Hadoop Developer Resume New York - Hire IT People

SUMMARY

Over 4+ years of overall experience in Big Data technologies and Hadoop ecosystem. Good understanding on Design, Development, Implementation, Testing and Deployment of Software Applications in Hadoop environment includes HDFS, HBase, Oozie, Pig, Hive, Kafka, Spark MapReduce. Experience installing, configuring, testing Hadoop ecosystem components.
Experience processing large amounts of structured and unstructured data, including integrating data from multiple sources.
Experienced in importing and exporting data from different sources using flumeandSqoopfrom RDBMS to HDFS and vice - versa.
Good understanding and noledge of NOSQL databases like HBase and Cassandra.
Proficient in using RDMS concepts with MySQL, Oracle, SQL-Server.
Collaborate with development and QA teams to maintain high-quality deployment.
Good noledge of spark architecture and workflow.
Experience in writing SQLs across different databases - Oracle, Teradata, Hive.
Experience in production support and troubleshooting.
Experience in developing end-to-end ETL processes involving both real-time and batch ingestion.
Performed systems analysis for several information systems documenting and identifying performance and administrative bottlenecks.
Experienced in optimizing the spark jobs and hive queries.
Job workflow scheduling and monitoring using tools likeOozie and AutoSys.

TECHNICAL SKILLS

PROFESSIONAL EXPERIENCE

Confidential, New York

Hadoop Developer

Responsibilities:

Responsible for data extraction and data ingestion from different data sources into Hadoop Data Lake by creating ETL pipelines using Pig, and Hive.
Involved to scooping data from various RDBMS to HDFS and HIVE and vice-versa.
Worked with systems engineering team to plan and deploy new Hadoop environments and expand existing Hadoop clusters and Experience in converting MapReduce applications to Spark.
Involved in all phases of Software Development Life Cycle (SDLC) and Worked on all activities related to the development, implementation, and support for Hadoop.
Created and maintained technical documentation for launching HADOOP Clusters and for executing Hive queries.
Experience in managing and reviewing Hadoop log files from web servers and integrated into HDFS using Flume.
Involved in converting Hive/SQL queries into Spark transformations using Spark RDD, Scala, and Python.
Experience in agile methodologies, daily scrum meetings, planning.

Confidential, Jersey City, New Jersey

Jr Hadoop Developer

Responsibilities:

Installed/Configured/Maintained Apache Hadoop clusters for application development and Hadoop tools like Hive, Pig, HBase, Zookeeper and Sqoop.
Experience in creating different type of Hive tables in and loading data into them.
Perform architecture design, data modeling, and implementation of Big Data platform and analytic applications for the consumer products.
Worked on setting up high availability for major production cluster Hadoop version updates using automation tools.
Performed unit testing the Scala code in Dev environment.
Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.
Implemented test scripts to support test driven development and continuous integration.
Used JIRA for issue tracking and overall documentation.

Confidential, New York

Hadoop Production Support

Responsibilities:

Handled importing of data from various sources, performed transformations using Pig and loaded data into HDFS and Extracted the data fromMySQLintoHDFSusing Sqoop.
Involved in loading data fromUNIXfile system toHDFS.
UsedSparkstream processing get data into in-memory, implementedRDDtransformations, actions to process as units.
Involved in the development of Spark Streaming application for faster testing and data processing from the data sources using Scala, Spark-SQL.
Created and workedSqoopjobs with incremental load to populateHiveExternal tables.
Managing and scheduling Jobs on a Hadoop Cloudera cluster usingOozieworkflows.

Confidential, New York

Intern-Hadoop Production Support

Responsibilities:

Hands-on experience in working with development teams for developing Hadoop applications and providing solutions.
Hands on experience in developing end-to-end solutions to analyze large data sets efficiently.
Hands on experience in designing and implementation of distributed application architecture.
Hands on experience in creating and implementing build and deployment procedures.
Performed tasks to ensure all production processed according to Change Management policies and procedures.
Participated in integration testing activities with QA team and the development team.
Experienced in job scheduling application like Autosys.