We provide IT Staff Augmentation Services!

Hadoop Developer Resume

5.00/5 (Submit Your Rating)

Chicago, IL

SUMMARY

  • Java certified professional with over 7 years of working experience in designing, development, building frameworks, testing, and implementation of Java/J2EE and Big Data related technologies.
  • Experience working in Hadoop development and administration and managing Hadoop cluster with components such as HDFS, Sqoop, Flume, Spark, Pig, Hive, Kafka, Zookeeper, etc.
  • Experience in Big Data technologies such as Pig, Hive, HDFS, Sqoop, Flume, HBase, Spark, Storm, Kafka, Yarn/MRv2, Solr, Oozie, Zookeeper, and Cassandra.
  • Experience in writing custom UDFs to extend functionality using for Pig and Hive.
  • Experience in working with Data Analysis using Pig and Hive.
  • Extensive experience working with Java/J2EE development.
  • Experience with importing and exporting data from RDBMS like DB2, MySQL, and Oracle into HDFS and teh other way around using Sqoop.
  • Extensive experience with job scheduling using Oozie workflow engine and Zookeeper.
  • In depth noledge in analyzing data using MapReduce, Pig, Latin, and Hive.
  • Experience with writing MapReduce jobs using Java, Spark, Pig, and Hive.
  • Good experience with Apache Kafka and with ingesting data into teh Spark Engine.
  • Experience with working on Spark projects with Spark streaming using Java.
  • In - depth experience with Cloudera Manager as teh monitoring tool for teh Hadoop ecosystem.
  • Experience with setting up teh Hadoop cluster ranging from a few nodes to much larger clusters.
  • Experience with monitoring teh health of teh NodeManagers and running scripts to manage teh health of teh node.
  • Good noledge on Batch Analytics, Data Ingestion, and Data Processing of in-stream data.
  • Strong background in working with Cloudera, Impala, and Spark.
  • Good experience working with teh Kafka messaging system.
  • Good understanding of Teradata, Zeppelin and SOLR.
  • Experience with working on NoSQL Databases such as HBase, MongoDB, and Cassandra.
  • Good understanding of RDBMS such as DB2, Oracle, and SQL server.
  • Familiarity with AWS solutions such as Elastic MapReduce, EC2, and EBS.
  • Troubleshooting problems on a day-to-day basis to manage teh performance issues within teh cluster.
  • Good noledge on ETL in Hadoop for Data extraction, transformation, and loading.
  • Extensive experience in Java, Shell scripting, and Relational Databases in writing Triggers and Stored Procedures.
  • Experience with Linux in with respect to administration, and security.
  • Experience with J2EE technologies such as Servlets, JSP, JDBC, EJB, and JMS.
  • Experience working on SOAP and REST based webservices.
  • Good understanding with Java frameworks like Spring, and Struts MVC and ORM framework like Hibernate.
  • Good experience using IDEs such as Eclipse, IntelliJ, and NetBeans.
  • Ability to conduct Proof of Concept on technologies.
  • Ability to analyze functional and non-functional requirements and constantly seeking to get a better clarification of teh requirements.
  • Experience in production support, as well as monitoring and scheduling of jobs.
  • Ability to collaborate and communicate TEMPeffectively with team members to deliver strong results.

TECHNICAL SKILLS

Hadoop: HDFS, YARN, MapReduce, Hive, Pig, Latin, Flume, Sqoop, Mahout, KafkaSpark, Cloudera, HortonWorks.

Languages: Java, Java/J2EE, Python, Scala, Shell Script, SQL and PL/SQL.

Web Technologies: HTML5, CSS3, JavaScript

Query Languages: SQL, HiveQL, PL/SQL, Pig

Databases: Mongo DB, HBase, Cassandra, Oracle, DB2.

Version Control Tools: JIRA, CVS, SVN, and GitHub.

Methodologies: Agile, Scrum, Waterfall Model.

Operating Systems: Windows 8/ 7/ XP/ 2000/NT/98/95, MS-DOS, UNIX.

PROFESSIONAL EXPERIENCE

Confidential, Chicago, IL

Hadoop Developer

Responsibilities:

  • Developed and Deployed Hadoop Cluster using Pig, Hive, Hbase, Oozie, Sqoop, Spark, Impala, and Kafka.
  • Worked with Sqoop to import and export data into HDFS, HIVE, and HBASE.
  • Implemented MapReduce jobs in Java for data processing.
  • Used Sqoop imports to load and transform large sets of data into HDFS from relational databases.
  • Experience in writing Map Reduce programs for Data Analysis using Java.
  • Involved in integrating Apache Kafka and Apache Storm.
  • Performed data transformation and a few pre-aggregations before storing teh data onto HDFS by using Pig.
  • Created structured data from a pool of unstructured data using Spark.
  • Involved in creating Spark applications using spark shell and pyspark.
  • Performed in-memory computing capacity of Spark to perform procedures such as text analysis and processing using Scala.
  • Experience working with Spark Streaming and divided data into different branches for batch processing through teh Spark engine.
  • Worked on parallel processing using MapReduce and Spark.
  • Created Hive UDFs using Java.
  • Loading data from disparate data sets using Flume and Sqoop.
  • Worked on job scheduling using Oozie workflow engine.
  • Worked on installation, support, and monitoring of Hadoop clusters using Cloudera manager.
  • Worked with teh Cloudera distributions
  • Worked with Apache Kafka to collect, aggregate and move large amounts of data from application servers.
  • Worked on integrating Cassandra with Elastic Search and Hadoop.
  • Involved in creating HBASE tables to store data from UNIX and NoSQL.
  • Troubleshooting and debussing runtime issues in teh Hadoop ecosystem.
  • Involved in integrating algorithms into production system by working with teh engineering team.

Environment: Java, Hadoop, MapReduce, Pig, Hive, Oozie, Sqoop, Spark, Cloudera, Kafka, Cassandra, HBase.

Confidential, Minneapolis, MN

Hadoop Developer

Responsibilities:

  • Development of a project of Cloudera Hadoop cluster implementation.
  • Created MapReduce programs to parse raw data and to populate staging tables.
  • Used Hive and Pig to analyze teh data.
  • Worked on expanding teh existing Hadoop clusters by working closely with teh engineering team to deploy new Hadoop environments.
  • Developed Spark scripts using Scala and accessed hive tables to improve teh quickness of data processing.
  • Used Oozie workflow in to create job flows.
  • Involved in performance tuning to support using HDFS and Hive to support teh application.
  • Used Spark streaming with Kafka to perform Real-time processing.
  • Worked with Spark and Kafka for Real time Data Ingestion.
  • Querying data using SparkSql over teh Spark engine.
  • Wrote some high-performance code using Spark as well as implanting Flume processes.
  • Used Cloudera manager to manage and monitor Hadoop cluster.
  • Worked on developing UDFs in hive.
  • Worked with relational databases such as Oracle and DB2.
  • Used Sqoop to load data from MySQL to HBase.
  • Used Flume and Sqoop for data integration.
  • Working with teh Systems Engineers in assessing teh hardware and finding ways to improve performance.
  • Working with teh end users in developing teh code and testing teh application.
  • Conforming to teh coding standards specified to develop teh code and ensuring quality.
  • Collaborating with project and product managers in order to implement teh deliverables.

Environment: Hadoop, MapReduce, HDFS, Sqoop, Pig, Hive, Spark, Oozie, Java, HBase, Scala, Shell Script.

Confidential, Miami, FL

Hadoop Developer/Administrator

Responsibilities:

  • Worked with business analysts to understand teh requirements.
  • Worked on Cloudera manager for managing and monitoring teh Hadoop cluster.
  • Involved with defining and managing Hadoop schedules and job flows.
  • Experience with working on data pipelining for ingesting data into HDFS for analysis using Flume and Sqoop.
  • Managing teh cluster and scheduling workflows using Zookeeper, Oozie, and operational services.
  • Developed a reporting application which can generate reports from HBase table by using Spark SQL.
  • Worked on analyzing large sets of data by developing Hive queries and Pig scripts.
  • Processing large volumes of data by running MapReduce jobs.
  • Performed pre-processing using Pig and Hive.
  • Worked with NoSQL Databases like MongoDB and HBase.
  • Experience with developing Spark SQL queries to load data into HDFS and to run querries on top.
  • Worked on loading data from UNIX file systems to HDFS.
  • Worked on creating custom UDFs to extend functionalities of Hive and Pig.
  • Experience with Apache Kafka for collecting large volumes of data, aggregating and moving data from application servers.
  • Designing monitoring tools dat are important for teh Hadoop ecosystem.
  • Maintaining teh recovery of teh HDFC systems.
  • Conducting regular testing to improve teh performance.
  • Troubleshooting issues related to teh execution of MapReduce jobs.
  • Monitoring health of teh cluster on a regular basis and work on performance tuning.
  • Involved in teh implementation of Data Marts in Hadoop.
  • Taking care of teh tickets in teh Hadoop ecosystem.

Environment: Hadoop, MapReduce, Hive, Sqoop, Spark, Cloudera Kafka, Pig, Yarn, HBase, MongoDB, Cassandra, Java, Junit.

Confidential, Grand Prairie, TX

Hadoop Administrator

Responsibilities:

  • Installing configuring, and maintaining Hadoop cluster using technologies such as HBase, Pig, Hive, Spark, and MapReduce.
  • Developed MapReduce jobs using Java to transform and process data.
  • Experience with installing and configuring Hive and developing Hive UDFs.
  • Worked on building Cassandra clusters on AWS.
  • Involved in setting up POC Hadoop cluster on teh Amazon EC2 platform.
  • Worked on integrating Hadoop with other tools such as BI and EDW.
  • Experience with extracting data from different sources and loading them onto HDFS for processing.
  • Testing Hive, HDFS, and MapReduce for teh users.
  • Worked on configuration and capacity planning of MapReduce, HBase, and Yarn.
  • Worked on fixing bugs and issues from teh customers from time to time.
  • Experience with developing Pig code in order to load, filer and store data.
  • Developed workflows for Hive and Sqoop jobs using Oozie.
  • Debugging several development/production jobs dat are slow-running.
  • Worked with teh development team to gain feedback on teh design.
  • Setting up teh test and production environment.
  • Involved in teh maintenance of Cloudera Hadoop clusters.
  • Working with teh Linux administrator to administer teh Operating Systems and teh hardware.
  • Working closely with teh business analyst teams and providing support for development.
  • Worked of creating and documenting ETL pipelines.
  • Reducing time taken to query in large volumes of data.

Environment: Hadoop, Amazon EC2, Oozie, Sqoop, Pig, Spark, HDFS, Shell scripting, HBase, Cassandra.

Confidential

Java Developer

Responsibilities:

  • Worked on teh Software Development Life Cycle from requirement gathering, coding, to implementation.
  • Used teh Agile methodology and was involved in user-story based development.
  • Experience with working on various design patterns such as Singleton, Factory, Adapter, and Session Facade.
  • Worked on web-development using teh Struts Framework.
  • Used teh Struts validation framework for client-side validation.
  • Implemented teh RestFul web services in order to communicate with other applications.
  • Experience with developing SOAP and RestFul webservices with JSON and XML.
  • Used MQ in order to connect with teh back-end application using RestFul webservices and JSON.
  • Involved in Front-End development using HTML, CSS, and Javascript.
  • Involved in writing JSPs and Servlets.
  • Wrote stored procedure using PL/SQL.
  • Experience working with JDBC and SQL for teh Database connectivity.
  • Used GIT as teh version control tool.
  • Involved in performance tuning of teh application.
  • Involved in maintenance and production support.
  • Worked closely with teh deployment team during teh release management and deployment process.
  • Troubleshooting to resolve any software related issues and performing root cause analysis.

Environment: Java, J2EE, Web services, Stuts, JSP, UML, Oracle PL/SQL, Git.

We'd love your feedback!