We provide IT Staff Augmentation Services!

Hadoop Developer Resume

3.00/5 (Submit Your Rating)

Bentoville, AR

SUMMARY

  • Having 8+ years’ experience using object oriented design methodologies (OOD), experience in core Java, Big Data and allied technologies.
  • Experience in developing applications using Java and Big data solutions.
  • 3+ years in scalable, distributed software systems like Hadoop (MR1 & YARN), HDFS, Pig, MapReduce, Hive, HBase, Flume and Sqoop.
  • 2+ years of experience in Real time streaming frameworks like Kafka, Storm and Spark.
  • Experience with various Hadoop Distributions like CDH, HDP and Apache.
  • Led end - end design, development and production deployment for critical projects like real time and Batch Data replication.
  • 2+ years of experience in NoSQL databases like MongoDB and HBase.
  • 1+ years of experience in AWS (S3, EC2).
  • 5+ years of experience in Agile Methodology.
  • Solid understanding of relational database concepts.
  • Solid understanding of data warehouse star schema concepts.
  • Experience in creating workflows sessions in Workflow Manager and running the workflows in Workflow Monitor and analyzing them.
  • Expertise in unit testing using JUnit. Experience in error logging and debugging using Log4J.
  • Goal oriented, organized, team player with good interpersonal skills; thrives well within group environment as well as individually.
  • Strong business and application analysis skills with excellent communication and professional abilities.

TECHNICAL SKILLS

Big Data Stack: Hadoop, Hive, Sqoop, Pig, HBase (NoSQL), Apache Spark (Spark SQL, Spark streaming) and Oozie

Programming Languages: Java, Scala - Beginner

Databases: PostgreSQL, MS SQL server and Oracle

IDE’s & Utilities: Eclipse, SQL DeveloperOperating Systems UNIX, LINUX and Windows

PROFESSIONAL EXPERIENCE

Confidential, Bentoville, AR

Hadoop Developer

Responsibilities:

  • Lead team during initial set up of Hadoop platform, designed the data ingestion and validation approach with ETL flow.
  • Worked on extracting data from Oracle database and load to Hive database
  • Worked extensively on optimizing transformations for better performance
  • Involved in designing approach to handle cursors, functions, stored procedures in PL/SQL to Hive Query Language
  • Developed extraction logic using Sqoop scripts to move data from relational databases to HDFS
  • Developed complex transformations using Hive QL to build aggregate/summary tables
  • Optimized the performance of Hive Queries by looking into the Explain plan and also tuning various hive parameters.
  • Developed UDF’s to implement functions that were not present in Hive
  • Handled Update operations using Hive Queries
  • Involved in using HCATALOG to access Hive table metadata from Map Reduce or Pig code.
  • Performing data validation
  • Developed deployment scripts for Development, QA and Production systems through SVN.
  • Developed workflows using Oozie
  • Expertise in working in Agile environment

Environment: Hadoop, HDFS, Map Reduce, Hive, Oozie, SQL Developer, TOAD, Oracle, PL/SQL, Tortoise, SVN, Eclipse, JAVA, Shell scripting, Spark SQL, Scala, Unix and Tableau

Confidential, Falls Church, Virginia

Hadoop Developer

Responsibilities:

  • Designed an approach for Data Validation and Data Ingestion Framework.
  • Configured Pentaho DI with Hadoop to run the ETL process
  • Worked on validating and converting raw data from various streams like Email, Instant messaging, IP
  • Phone, Desktop Video and Immersive Video to csv files
  • Develop Pentaho jobs to load the data from source systems to HDFS
  • Develop transformation logic using Hive Queries to build dimension and fact tables
  • Developed schema in HBase for faster scans
  • Used HBase for storing aggregated data used for reporting
  • Worked on Unit testing by creating test data and comparing expected results
  • Developed deployment scripts for production release.
  • Participated in Daily scrum calls and track day to day activities using Rally
  • Expertise in working in Agile environment.

Environment: Hadoop, MapReduce, Yarn, Hive, Pig, HBase, Oozie, Sqoop, Flume, Oracle 11g, Core Java Cloudera HDFS, Eclipse.

Confidential

Hadoop Developer

Responsibilities:

  • Involved in all phases of the Big Data Implementation including requirement analysis, design, development, building, testing, and deployment of Hadoop cluster in fully distributed mode
  • Created Linux and Python Scripts to automate the daily ingestion of raw data
  • Processed the raw data using Hive jobs and scheduling them in Crontab
  • Developed HIVE UDFs to get the MDK and GeoIp values
  • Moved data to appropriate partition based on record-level timestamp(as we have more than one day’s worth of data in log files)
  • Compressed transformed/enriched data files with bzip2Codec
  • Experience in Daily production support to monitor and trouble shoots Hadoop/Hive jobs
  • Developed Housekeeping process purge the old data from Edge node and HDFS.
  • Implemented two different process for Internal and External Weblogs
  • Manage and review Hadoop log files.
  • Support/Troubleshoot hive programs running on the cluster
  • Involved in fixing issues arising out of duration testing
  • Handling structured, semi structured and unstructured data
  • Automated the History and Purge Process
  • Involved in the process of migrating to the new cluster.

Environment: Hadoop 2.x, Hive 0.13.1, Python, Unix Scripts, HDP 2.3, Redhat Linux

Confidential

Java/J2EE Developer

Responsibilities:

  • Responsible for the development of new requirements for the presentation layer developed according to FSA style Guide andUSDAstyle guide.
  • Preparing technical specifications, work assignments, coding and unit testing.
  • Responsible for the development of Assessment Calculation reports module.
  • Junit used for unit testing of the application.
  • Maven build is used to build the application on Eclipse 3.2 IDE.
  • Responsible for implementing reports module by using Springs MVC FrameWork.
  • Responsible for reviewing and approving the projects related documents like Design document, Database design documents and so on.
  • Responsible for designing the process configuration and event representation.
  • Responsible for the requirement gathering from the customer and finalizing the features.
  • Responsible for updating project status to the senior project manager in a regular interval.

Environment: JDK 6.0, EJB, JSP, Net beans IDE 6.0, JBoss, Dojo, Web Services, VB Script, Apache POI, Hibernate 3.0, JQuery.

Confidential

Java/J2EE Developer

Responsibilities:

  • Involved in walking through the functional requirements and estimating the effort for delivering these requirements.
  • Developed the Backend code, which has the business logic and interacts with the database.
  • Unit Testing and delivering a quality piece of code.
  • Perform Build and deployment activities by using Maven build.
  • Involved in Database schema design, developed stored procedures.

Environment: Java 1.6, Amdocs Smart Client Framework 8.0, Log 4j, Quality Control, MKS

We'd love your feedback!