Hadoop Developer Resume
Chicago, IL
SUMMARY
- Java certified professional with over 7 years of working experience in designing, development, building frameworks, testing, and implementation of Java/J2EE and Big Data related technologies.
- Experience working in Hadoop development and administration and managing Hadoop cluster with components such as HDFS, Sqoop, Flume, Spark, Pig, Hive, Kafka, Zookeeper, etc.
- Experience in Big Data technologies such as Pig, Hive, HDFS, Sqoop, Flume, HBase, Spark, Storm, Kafka, Yarn/MRv2, Solr, Oozie, Zookeeper, and Cassandra.
- Experience in writing custom UDFs to extend functionality using for Pig and Hive.
- Experience in working with Data Analysis using Pig and Hive.
- Extensive experience working with Java/J2EE development.
- Experience with importing and exporting data from RDBMS like DB2, MySQL, and Oracle into HDFS and the other way around using Sqoop.
- Extensive experience with job scheduling using Oozie workflow engine and Zookeeper.
- In depth knowledge in analyzing data using MapReduce, Pig, Latin, and Hive.
- Experience with writing MapReduce jobs using Java, Spark, Pig, and Hive.
- Good experience with Apache Kafka and with ingesting data into the Spark Engine.
- Experience with working on Spark projects with Spark streaming using Java.
- In - depth experience with Cloudera Manager as the monitoring tool for the Hadoop ecosystem.
- Experience with setting up the Hadoop cluster ranging from a few nodes to much larger clusters.
- Experience with monitoring the health of the NodeManagers and running scripts to manage the health of the node.
- Good knowledge on Batch Analytics, Data Ingestion, and Data Processing of in-stream data.
- Strong background in working with Cloudera, Impala, and Spark.
- Good experience working with the Kafka messaging system.
- Good understanding of Teradata, Zeppelin and SOLR.
- Experience with working on NoSQL Databases such as HBase, MongoDB, and Cassandra.
- Good understanding of RDBMS such as DB2, Oracle, and SQL server.
- Familiarity with AWS solutions such as Elastic MapReduce, EC2, and EBS.
- Troubleshooting problems on a day-to-day basis to manage the performance issues within the cluster.
- Good knowledge on ETL in Hadoop for Data extraction, transformation, and loading.
- Extensive experience in Java, Shell scripting, and Relational Databases in writing Triggers and Stored Procedures.
- Experience with Linux in with respect to administration, and security.
- Experience with J2EE technologies such as Servlets, JSP, JDBC, EJB, and JMS.
- Experience working on SOAP and REST based webservices.
- Good understanding with Java frameworks like Spring, and Struts MVC and ORM framework like Hibernate.
- Good experience using IDEs such as Eclipse, IntelliJ, and NetBeans.
- Ability to conduct Proof of Concept on technologies.
- Ability to analyze functional and non-functional requirements and constantly seeking to get a better clarification of the requirements.
- Experience in production support, as well as monitoring and scheduling of jobs.
- Ability to collaborate and communicate effectively with team members to deliver strong results.
TECHNICAL SKILLS
Hadoop: HDFS, YARN, MapReduce, Hive, Pig, Latin, Flume, Sqoop, Mahout, KafkaSpark, Cloudera, HortonWorks.
Languages: Java, Java/J2EE, Python, Scala, Shell Script, SQL and PL/SQL.
Web Technologies: HTML5, CSS3, JavaScript
Query Languages: SQL, HiveQL, PL/SQL, Pig
Databases: Mongo DB, HBase, Cassandra, Oracle, DB2.
Version Control Tools: JIRA, CVS, SVN, and GitHub.
Methodologies: Agile, Scrum, Waterfall Model.
Operating Systems: Windows 8/ 7/ XP/ 2000/NT/98/95, MS-DOS, UNIX.
PROFESSIONAL EXPERIENCE
Confidential, Chicago, IL
Hadoop Developer
Responsibilities:
- Developed and Deployed Hadoop Cluster using Pig, Hive, Hbase, Oozie, Sqoop, Spark, Impala, and Kafka.
- Worked with Sqoop to import and export data into HDFS, HIVE, and HBASE.
- Implemented MapReduce jobs in Java for data processing.
- Used Sqoop imports to load and transform large sets of data into HDFS from relational databases.
- Experience in writing Map Reduce programs for Data Analysis using Java.
- Involved in integrating Apache Kafka and Apache Storm.
- Performed data transformation and a few pre-aggregations before storing the data onto HDFS by using Pig.
- Created structured data from a pool of unstructured data using Spark.
- Involved in creating Spark applications using spark shell and pyspark.
- Performed in-memory computing capacity of Spark to perform procedures such as text analysis and processing using Scala.
- Experience working with Spark Streaming and divided data into different branches for batch processing through the Spark engine.
- Worked on parallel processing using MapReduce and Spark.
- Created Hive UDFs using Java.
- Loading data from disparate data sets using Flume and Sqoop.
- Worked on job scheduling using Oozie workflow engine.
- Worked on installation, support, and monitoring of Hadoop clusters using Cloudera manager.
- Worked with the Cloudera distributions
- Worked with Apache Kafka to collect, aggregate and move large amounts of data from application servers.
- Worked on integrating Cassandra with Elastic Search and Hadoop.
- Involved in creating HBASE tables to store data from UNIX and NoSQL.
- Troubleshooting and debussing runtime issues in the Hadoop ecosystem.
- Involved in integrating algorithms into production system by working with the engineering team.
Environment: Java, Hadoop, MapReduce, Pig, Hive, Oozie, Sqoop, Spark, Cloudera, Kafka, Cassandra, HBase.
Confidential, Minneapolis, MN
Hadoop Developer
Responsibilities:
- Development of a project of Cloudera Hadoop cluster implementation.
- Created MapReduce programs to parse raw data and to populate staging tables.
- Used Hive and Pig to analyze the data.
- Worked on expanding the existing Hadoop clusters by working closely with the engineering team to deploy new Hadoop environments.
- Developed Spark scripts using Scala and accessed hive tables to improve the quickness of data processing.
- Used Oozie workflow in to create job flows.
- Involved in performance tuning to support using HDFS and Hive to support the application.
- Used Spark streaming with Kafka to perform Real-time processing.
- Worked with Spark and Kafka for Real time Data Ingestion.
- Querying data using SparkSql over the Spark engine.
- Wrote some high-performance code using Spark as well as implanting Flume processes.
- Used Cloudera manager to manage and monitor Hadoop cluster.
- Worked on developing UDFs in hive.
- Worked with relational databases such as Oracle and DB2.
- Used Sqoop to load data from MySQL to HBase.
- Used Flume and Sqoop for data integration.
- Working with the Systems Engineers in assessing the hardware and finding ways to improve performance.
- Working with the end users in developing the code and testing the application.
- Conforming to the coding standards specified to develop the code and ensuring quality.
- Collaborating with project and product managers in order to implement the deliverables.
Environment: Hadoop, MapReduce, HDFS, Sqoop, Pig, Hive, Spark, Oozie, Java, HBase, Scala, Shell Script.
Confidential, Miami, FL
Hadoop Developer/Administrator
Responsibilities:
- Worked with business analysts to understand the requirements.
- Worked on Cloudera manager for managing and monitoring the Hadoop cluster.
- Involved with defining and managing Hadoop schedules and job flows.
- Experience with working on data pipelining for ingesting data into HDFS for analysis using Flume and Sqoop.
- Managing the cluster and scheduling workflows using Zookeeper, Oozie, and operational services.
- Developed a reporting application which can generate reports from HBase table by using Spark SQL.
- Worked on analyzing large sets of data by developing Hive queries and Pig scripts.
- Processing large volumes of data by running MapReduce jobs.
- Performed pre-processing using Pig and Hive.
- Worked with NoSQL Databases like MongoDB and HBase.
- Experience with developing Spark SQL queries to load data into HDFS and to run querries on top.
- Worked on loading data from UNIX file systems to HDFS.
- Worked on creating custom UDFs to extend functionalities of Hive and Pig.
- Experience with Apache Kafka for collecting large volumes of data, aggregating and moving data from application servers.
- Designing monitoring tools that are important for the Hadoop ecosystem.
- Maintaining the recovery of the HDFC systems.
- Conducting regular testing to improve the performance.
- Troubleshooting issues related to the execution of MapReduce jobs.
- Monitoring health of the cluster on a regular basis and work on performance tuning.
- Involved in the implementation of Data Marts in Hadoop.
- Taking care of the tickets in the Hadoop ecosystem.
Environment: Hadoop, MapReduce, Hive, Sqoop, Spark, Cloudera Kafka, Pig, Yarn, HBase, MongoDB, Cassandra, Java, Junit.
Confidential, Grand Prairie, TX
Hadoop Administrator
Responsibilities:
- Installing configuring, and maintaining Hadoop cluster using technologies such as HBase, Pig, Hive, Spark, and MapReduce.
- Developed MapReduce jobs using Java to transform and process data.
- Experience with installing and configuring Hive and developing Hive UDFs.
- Worked on building Cassandra clusters on AWS.
- Involved in setting up POC Hadoop cluster on the Amazon EC2 platform.
- Worked on integrating Hadoop with other tools such as BI and EDW.
- Experience with extracting data from different sources and loading them onto HDFS for processing.
- Testing Hive, HDFS, and MapReduce for the users.
- Worked on configuration and capacity planning of MapReduce, HBase, and Yarn.
- Worked on fixing bugs and issues from the customers from time to time.
- Experience with developing Pig code in order to load, filer and store data.
- Developed workflows for Hive and Sqoop jobs using Oozie.
- Debugging several development/production jobs that are slow-running.
- Worked with the development team to gain feedback on the design.
- Setting up the test and production environment.
- Involved in the maintenance of Cloudera Hadoop clusters.
- Working with the Linux administrator to administer the Operating Systems and the hardware.
- Working closely with the business analyst teams and providing support for development.
- Worked of creating and documenting ETL pipelines.
- Reducing time taken to query in large volumes of data.
Environment: Hadoop, Amazon EC2, Oozie, Sqoop, Pig, Spark, HDFS, Shell scripting, HBase, Cassandra.
Confidential
Java Developer
Responsibilities:
- Worked on the Software Development Life Cycle from requirement gathering, coding, to implementation.
- Used the Agile methodology and was involved in user-story based development.
- Experience with working on various design patterns such as Singleton, Factory, Adapter, and Session Facade.
- Worked on web-development using the Struts Framework.
- Used the Struts validation framework for client-side validation.
- Implemented the RestFul web services in order to communicate with other applications.
- Experience with developing SOAP and RestFul webservices with JSON and XML.
- Used MQ in order to connect with the back-end application using RestFul webservices and JSON.
- Involved in Front-End development using HTML, CSS, and Javascript.
- Involved in writing JSPs and Servlets.
- Wrote stored procedure using PL/SQL.
- Experience working with JDBC and SQL for the Database connectivity.
- Used GIT as the version control tool.
- Involved in performance tuning of the application.
- Involved in maintenance and production support.
- Worked closely with the deployment team during the release management and deployment process.
- Troubleshooting to resolve any software related issues and performing root cause analysis.
Environment: Java, J2EE, Web services, Stuts, JSP, UML, Oracle PL/SQL, Git.
