We provide IT Staff Augmentation Services!

Hadoop Developer Resume

5.00/5 (Submit Your Rating)

Birmingham, AlabamA

SUMMARY

  • 8+ years of IT Experience in systems development, databases & analytics, with 3 years of comprehensive experience as Hadoop Developer.
  • Expertise in Big data, Hadoop, SQL,PL/SQL and various components in Hadoop such as HDFS, MR2, YARN, Spark, PIG, Hive, Sqoop, HBase, Cloudera Manager, Zoo keeper, Oozie, Kafka, Hue, CDH5, & HDP 2.x.Expertise in writing Hadoop Jobs for analyzing data using MapReduce, Hive, & Pig.
  • Working experience on Cloudera, Horton Works Hadoop distribution.
  • Extensive experience in developing PIG Latin Scripts and using Hive Query Language for data analytics.
  • Experience in Installation, sizing & configuration of Hadoop eco system.
  • Transformed date related data into application compatible format by developing apache Pig UDFs.
  • Successfully migrated Legacy application to Big Data application using Hive/Pig/HBase in Production level.
  • Expert in working with Hive data warehouse tool­creating tables, data distribution by implement­ing partitioning and bucketing, writing and optimizing the HiveQL queries.
  • Experience in importing and exporting data using Sqoop to HDFS from Relational Database Systems.
  • Hands - on Experience in installing, configuring and maintaining the Hadoop clusters.
  • Experience in designing both time driven and data driven automated workflows using Oozie.
  • Good understanding of NoSQL Data bases like HBase.
  • Experience in AWS - S3, EC2, Redshift.
  • Used HiveQL to do analysis on the data and identify different correlations
  • Experience in developing custom UDFs for Pig and Hive to in corporate methods and functionality of Python into Pig Latin and HiveQL.
  • Written MapReduce programs in Python with the Hadoop streaming API.
  • Good Knowledge of Data Profiling using Informatica Data Explorer.
  • Extensive experience in building ETL Design and Development.
  • Good understanding of Project Management Knowledge Areas and Process groups.
  • Experience working in an iterative, agile software lifecycle with strong ability to estimate/scope the development of projects.
  • Well versed in OLTP Data Modeling and Strong knowledge of Entity-Relationship concepts.
  • Experience in Data Cleaning andData Preprocessing usingPython Scripting.
  • Good experience in all the phases of Software Development Life Cycle (Analysis of requirements, Design, Development, Verification and Validation, Deployment).
  • Development, Verification and Validation, Deployment).
  • Have strong Database knowledge, good in PL/SQL programming and RDBS concepts.
  • Extensively involved in creating Oracle SQL queries, PL/SQL Stored Procedures, Functions, Packages, Triggers and Cursors with Query optimizations as part of ETL Development process.
  • Knowledge on Handling Hive queries using Spark SQL that integrate Spark environment.
  • Hands on experience of UNIX and shell scripting to automate scripts.
  • Worked on FileZilla and WinScp tools for transferring files to UNIX environments.
  • Good skills in JavaMultithreading, Exception Handling, J2EE, JSP, Servlets, Struts, Spring, Hibernate, JDBC, SOAP, WSDL, JSON, JQuery, Java Script, CSS and HTML.
  • Developed applications using Spring Framework and implemented spring modules like core container module, application context module, Aspect oriented module (AOP Module), JDBC Module, ORM. Module and web module

TECHNICAL SKILLS

Hadoop/Big Data: HBase, HDFS, Hive, Hue, Impala, Oozie, Sentry, Solr, Spark, Sqoop 2, YARN, ZooKeeper, MapReduce, PIG, KMS, Flume, Hue, Cloudera Manager, Apache Ambari, HCatalog, Knox and Ranger.

Operating Systems: Windows, Ubuntu, RedHat Linux, Unix, c#

Java & J2EE Technologies: Core Java, Servlets, JSP, JDBC, Talend

Frameworks: Hibernate, Aws, kafka

Databases/Database Languages: Oracle 12c/11g/10g/9i, MySQL, DB2, SQL Server, SQL, HQL, NoSQL (HBase), Spark.

Web Technologies: JavaScript, HTML, XML, REST, CSS

Programming Languages: Java, Unix shell scripting, COBOL, CICS, JCL

IDE’s: Eclipse, Rational Web Application Developer, NetBeans, TextPad

App/Web Servers: Apache Tomcat Server, Apache / IBM HTTP Server, WebSphere Application Server 6.1/7.0

Methodologies: Agile, Waterfall model, Spiral model, SDLC

PROFESSIONAL EXPERIENCE

Confidential, Birmingham, Alabama

Hadoop Developer

Responsibilities:

  • Working on the development of a web application and Spring batch applications. The web application allows the customers to sign up and get the cellular and music services.
  • Tools: MySQL, Tomcat Server, Mybatis, Spring MVC, REST, AWS (Amazon Web Services)
  • Working on the development of User Interface
  • Tools: Angular JS, Backbone JS, java script, velocity
  • Working on the mobile payment functionality using PayPal, Angular JS and Spring MVC
  • Have been involved in Spring Integratio
  • Have been involved in the building and deployment of the applications using Ant build.
  • Involved in fixing the production bugs and also involved in the deployment process.
  • Have been working on Spring Batch applications to make sure the customer cellular and music services gets renewed Spring Batch
  • Involved in deploying the applications in AWS.
  • Proficiency in Unix/Linux shell commands.
  • Maintains the EC2 (Elastic Computing Cloud) and RDS (Relational Database Services) in amazon web services.
  • Created RESTful web services interface for supporting XML message transformation.
  • Developed Junit test case using TestNG.
  • Involved in designing the web applications and I closely work with architect.
  • Mainly working on handling of BigData Analytics and infrastructure of Hadoop, MapReduce
  • Got good experience with NoSQL database.

Environment: Hadoop (CDH), MapReduce, HDFS, Hive, Pig, Sqoop, Flume, Oozie, Java, SQL, Kafka, Cassandra

Confidential, Chino Hills, CA

Hadoop Developer

Responsibilities:

  • Setup Hadoop cluster onAmazon EC2using whirr for POC.
  • Worked on analysingHadoop clusterand different big data analytic tools includingPigHbasedatabase andSqoop, developed using working on a talend .
  • Responsible for buildingscalable distributed data solutionsusing Hadoop
  • Installed and configuredFlume Hive Pig Sqoop HBaseon the Hadoop cluster.
  • Managing and scheduling Jobs on a Hadoop cluster.
  • Implemented nine nodesCDH3Hadoop cluster on Red hat LINUX.
  • Worked on installing cluster commissioning decommissioning of datanode namenode recovery capacity planning and slots configuration.Using Spark to developing the application.
  • Resource management of HADOOP Cluster including adding/removing cluster nodes for maintenance and capacity needs,using Talend and . using c#.
  • Involved in loading data fromUNIX file systemtoHDFS.
  • CreatedHBase tablesto store variable data formats of PII data coming from different portfolios.
  • Implemented best income logic usingPigscripts.
  • Developed Microservices & APIs using Spring Boot and Used Apache Kafka cluster as messaging system between the APIs and Microservices.
  • Implemented test scripts to support test driven development and continuous integration.
  • Responsible to manage data coming from different sources.
  • Installed and configuredHiveand also writtenHive UDFs.
  • Experienced on loading and transforming of large sets of structured semi structured and unstructured data.
  • Cluster coordination services throughZookeeper.
  • Good Knowledge about scalable, secure cloud architecture based on Amazon Web Services (leveraging AWS cloud services: EC2, Cloud Formation, VPC, S3.
  • Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop MapReduce, HDFS, HBase, Hive, Sqoop, Pig, Zookeeper, Storm, Spark, Kafka and Flume.
  • Implemented multi-tiered architecture using both Microservices and Monolithic architecture.
  • Development of micro services using Dropwizard and Spring Boot. UI implementation using AngularJS 1.x.
  • Worked with team of developers on Python applications for RISK management.
  • DevelopedPython/ Django application for Google Analytics aggregation and reporting
  • Used Django configuration to manage URLs and application parameters.
  • UsedPythonscripts to update content in the database and manipulate files.
  • GeneratedPythonDjango Forms to record data of online users.
  • Usedpythonand Django creating graphics, XML processing, data exchange and business logic implementation.
  • Exported the analysed data to the relational databases usingSqoopfor visualization and to generate reports for the BI team.
  • Analysed large amounts of data sets to determine optimal way to aggregate and report on it.
  • Supported in setting up QA environment and updating configurations for implementing scripts withPigandSqoop.

Environment: Hadoop HDFS Hive Flume HBase Sqoop PIG Java JDK 1.6 Eclipse MySQL, Spring boot, Web Frameworks Spring (MVC, Core), talend, Dropwizard, c#, python,Spray-Can (Scala) and Ubuntu Zookeeper Amazon EC2 SOLR., Spark, kafka, Oozie

Confidential, San Mateo, CA

Hadoop Developer

Responsibilities:

  • Involved to provide architect, design, develop and testing services for sub-system components within the data aggregation infrastructure
  • Installed/Configured/Maintained Apache Hadoop clusters for application development and Hadoop tools like Hive, Pig, HBase, Flume and Sqoop.
  • Performed system administration activities on Linux, CentOs & Ubuntu. • Developed Java Map/Reduce job for Trip Calibration, Trip summarization and data filtering.
  • Developed Hive UDFs for rating aggregation.
  • Handled importing of data from various data sources, performed transformations using Hive, MapReduce, and loaded data into HDFS
  • Extracted the data from Teradata into HDFS using Sqoop.
  • Analyzed the data by performing Hive queries and running Pig scripts to know user behavior like shopping Enthusiasts, travelers, music lovers etc.
  • The patterns analyzed are exported back into Teradata using Sqoop.
  • Continuous monitoring and managing the Hadoop cluster through Cloudera Manager.
  • Installed Oozie workflow engine to run multiple Hive. • Monitoring workload, job performance and capacity planning using Cloudera Manager.
  • Developed Hive queries to process the data and generate the data cubes for visualizing
  • Importing and exporting data into HDFS and Hive using Sqoop.
  • Experienced in defining job flows.
  • Writing shell scripts for manipulating data.
  • Experienced in managing and reviewing Hadoop log files.
  • Responsible to manage data coming from different sources.
  • Used Oozie tool for job scheduling

Environment: Hadoop, MapReduce, HDFS, Java 6, Hadoop distribution, Apache Hadoop 1.0.1, MapReduceHDFS, CentOS, Zookeeper, Sqoop, Hive, Pig, Oozie, Java, Eclipse, Amazon EC2, JSP, Servlets, Oracle.

Confidential

Jr. Hadoop Developer

Responsibilities:

  • Worked closely with the Development Team in the design phase and developed use case diagrams using Rational Rose.
  • Responsible for building scalable distributed data solutions using Hadoop.
  • Installed and configured Hive, Pig, Sqoop, Flume and Oozie on the Hadoop cluster.
  • Setup and benchmarked Hadoop/HBase clusters for internal use.
  • Developed Simple to complex Map/reduce Jobs using Hive and Pig.
  • Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
  • Handled importing of data from various data sources, performed transformations using Hive, Map-Reduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop.
  • Analyzed the data by performing Hive queries and running Pig scripts to study customer behavior.
  • Used UDF's to implement business logic in Hadoop.
  • Implemented business logic by writing UDFs in Java and used various UDFs from Piggybanks and other sources.
  • Continuous monitoring and managing the Hadoop cluster using Cloudera Manager.
  • Developed Map-Reduce programs in Java for Data Analysis.
  • Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
  • Developed HQL for the analysis of semi structured data.
  • Handled the installation and configuration of a Hadoop cluster.
  • Build and maintained scalable data pipelines using the Hadoop ecosystem and other open source components like Hive, and Cassandra instead of HBase.
  • Used Hive and created Hive tables and involved in data loading and writing Hive UDFs.
  • Used Sqoop to import data into HDFS and Hive from other data systems.
  • Handle the data exchange between HDFS and different web sources using Flume and Sqoop
  • Installed Kafka on Hadoop cluster and configured producer and consumer coding part in java to establish connection

Environment: Hadoop (CDH), MapReduce, HDFS, Hive, Pig, Sqoop, Flume, Oozie, Java, SQL, Kafka, Cassandra.

Confidential

JAVA Developer

Responsibilities:

  • Installation, Configuration & Upgrade of Solaris and Linux operating system.
  • Actively participated in requirements gathering, analysis, design, and testing phases
  • Designed use case diagrams, class diagrams, and sequence diagrams as a part of Design Phase
  • Developed the entire application implementing MVC Architecture integrating JSF with Hibernate and Spring frameworks.
  • Developed the Enterprise Java Beans (Stateless Session beans) to handle different transactions such as online funds transfer, bill payments to the service providers.
  • Implemented Service Oriented Architecture (SOA) using JMS for sending and receiving messages while creating web services
  • Developed XML documents and generated XSL files for Payment Transaction and Reserve Transaction systems.
  • Developed SQL queries and stored procedures.
  • Developed Web Services for data transfer from client to server and vice versa using Apache Axis, SOAP and WSDL.
  • Used JUnit Framework for the unit testing of all the java classes.
  • Implemented various J2EE Design patterns like Singleton, Service Locator, DAO, and SOA.
  • Worked on AJAX to develop an interactive Web Application and JavaScript for Data Validations.
  • Developed the application under JEE architecture, developed Designed dynamic and browser compatible user interfaces using JSP, Custom Tags, HTML, CSS, and JavaScript.
  • Deployed & maintained the JSP, Servlets components on Web logic 8.0
  • Developed Application Servers persistence layer using, JDBC, SQL, Hibernate.
  • Used JDBC to connect the web applications to Data Bases.
  • Implemented Test First unit testing framework driven using Junit.
  • Developed and utilized J2EE Services and JMS components for messaging communication in Web Logic.
  • Configured development environment using Web logic application server for developer’s integration testing.

Environment: Java/J2EE, SQL, Oracle 10g, JSP 2.0, EJB, AJAX, Java Script, Web Logic 8.0, HTML, JDBC 3.0, XML, JMS, log4j, Junit, Servlets, MVC, My Eclipse

We'd love your feedback!