Hadoop Developer Resume
Bentoville, AR
SUMMARY
- Having 8+ years’ experience using object oriented design methodologies (OOD), experience in core Java, Big Data and allied technologies.
- Experience in developing applications using Java and Big data solutions.
- 3+ years in scalable, distributed software systems like Hadoop (MR1 & YARN), HDFS, Pig, MapReduce, Hive, HBase, Flume and Sqoop.
- 2+ years of experience in Real time streaming frameworks like Kafka, Storm and Spark.
- Experience with various Hadoop Distributions like CDH, HDP and Apache.
- Led end - end design, development and production deployment for critical projects like real time and Batch Data replication.
- 2+ years of experience in NoSQL databases like MongoDB and HBase.
- 1+ years of experience in AWS (S3, EC2).
- 5+ years of experience in Agile Methodology.
- Solid understanding of relational database concepts.
- Solid understanding of data warehouse star schema concepts.
- Experience in creating workflows sessions in Workflow Manager and running the workflows in Workflow Monitor and analyzing them.
- Expertise in unit testing using JUnit. Experience in error logging and debugging using Log4J.
- Goal oriented, organized, team player with good interpersonal skills; thrives well within group environment as well as individually.
- Strong business and application analysis skills with excellent communication and professional abilities.
TECHNICAL SKILLS
Big Data Stack: Hadoop, Hive, Sqoop, Pig, HBase (NoSQL), Apache Spark (Spark SQL, Spark streaming) and Oozie
Programming Languages: Java, Scala - Beginner
Databases: PostgreSQL, MS SQL server and Oracle
IDE’s & Utilities: Eclipse, SQL DeveloperOperating Systems UNIX, LINUX and Windows
PROFESSIONAL EXPERIENCE
Confidential, Bentoville, AR
Hadoop Developer
Responsibilities:
- Lead team during initial set up of Hadoop platform, designed the data ingestion and validation approach with ETL flow.
- Worked on extracting data from Oracle database and load to Hive database
- Worked extensively on optimizing transformations for better performance
- Involved in designing approach to handle cursors, functions, stored procedures in PL/SQL to Hive Query Language
- Developed extraction logic using Sqoop scripts to move data from relational databases to HDFS
- Developed complex transformations using Hive QL to build aggregate/summary tables
- Optimized the performance of Hive Queries by looking into the Explain plan and also tuning various hive parameters.
- Developed UDF’s to implement functions that were not present in Hive
- Handled Update operations using Hive Queries
- Involved in using HCATALOG to access Hive table metadata from Map Reduce or Pig code.
- Performing data validation
- Developed deployment scripts for Development, QA and Production systems through SVN.
- Developed workflows using Oozie
- Expertise in working in Agile environment
Environment: Hadoop, HDFS, Map Reduce, Hive, Oozie, SQL Developer, TOAD, Oracle, PL/SQL, Tortoise, SVN, Eclipse, JAVA, Shell scripting, Spark SQL, Scala, Unix and Tableau
Confidential, Falls Church, Virginia
Hadoop Developer
Responsibilities:
- Designed an approach for Data Validation and Data Ingestion Framework.
- Configured Pentaho DI with Hadoop to run the ETL process
- Worked on validating and converting raw data from various streams like Email, Instant messaging, IP
- Phone, Desktop Video and Immersive Video to csv files
- Develop Pentaho jobs to load the data from source systems to HDFS
- Develop transformation logic using Hive Queries to build dimension and fact tables
- Developed schema in HBase for faster scans
- Used HBase for storing aggregated data used for reporting
- Worked on Unit testing by creating test data and comparing expected results
- Developed deployment scripts for production release.
- Participated in Daily scrum calls and track day to day activities using Rally
- Expertise in working in Agile environment.
Environment: Hadoop, MapReduce, Yarn, Hive, Pig, HBase, Oozie, Sqoop, Flume, Oracle 11g, Core Java Cloudera HDFS, Eclipse.
Confidential
Hadoop Developer
Responsibilities:
- Involved in all phases of the Big Data Implementation including requirement analysis, design, development, building, testing, and deployment of Hadoop cluster in fully distributed mode
- Created Linux and Python Scripts to automate the daily ingestion of raw data
- Processed the raw data using Hive jobs and scheduling them in Crontab
- Developed HIVE UDFs to get the MDK and GeoIp values
- Moved data to appropriate partition based on record-level timestamp(as we have more than one day’s worth of data in log files)
- Compressed transformed/enriched data files with bzip2Codec
- Experience in Daily production support to monitor and trouble shoots Hadoop/Hive jobs
- Developed Housekeeping process purge the old data from Edge node and HDFS.
- Implemented two different process for Internal and External Weblogs
- Manage and review Hadoop log files.
- Support/Troubleshoot hive programs running on the cluster
- Involved in fixing issues arising out of duration testing
- Handling structured, semi structured and unstructured data
- Automated the History and Purge Process
- Involved in the process of migrating to the new cluster.
Environment: Hadoop 2.x, Hive 0.13.1, Python, Unix Scripts, HDP 2.3, Redhat Linux
Confidential
Java/J2EE Developer
Responsibilities:
- Responsible for the development of new requirements for the presentation layer developed according to FSA style Guide andUSDAstyle guide.
- Preparing technical specifications, work assignments, coding and unit testing.
- Responsible for the development of Assessment Calculation reports module.
- Junit used for unit testing of the application.
- Maven build is used to build the application on Eclipse 3.2 IDE.
- Responsible for implementing reports module by using Springs MVC FrameWork.
- Responsible for reviewing and approving the projects related documents like Design document, Database design documents and so on.
- Responsible for designing the process configuration and event representation.
- Responsible for the requirement gathering from the customer and finalizing the features.
- Responsible for updating project status to the senior project manager in a regular interval.
Environment: JDK 6.0, EJB, JSP, Net beans IDE 6.0, JBoss, Dojo, Web Services, VB Script, Apache POI, Hibernate 3.0, JQuery.
Confidential
Java/J2EE Developer
Responsibilities:
- Involved in walking through the functional requirements and estimating the effort for delivering these requirements.
- Developed the Backend code, which has the business logic and interacts with the database.
- Unit Testing and delivering a quality piece of code.
- Perform Build and deployment activities by using Maven build.
- Involved in Database schema design, developed stored procedures.
Environment: Java 1.6, Amdocs Smart Client Framework 8.0, Log 4j, Quality Control, MKS
