We provide IT Staff Augmentation Services!

Sr. Hadoop Developer Resume

0/5 (Submit Your Rating)

Denver, CO

SUMMARY

  • Over 8 years of professional IT experience in all phases of Software Development Life Cycle which includes 4 years of experience in Big Data, Hadoop Development and Ecosystem Analytics in Banking and Retail industries.
  • Hands on experience in using Hadoop Ecosystem components like HDFS, Map Reduce, Hive, Pig, HBase, Zookeeper, Sqoop, Flume, Kafka, Oozie and Spark.
  • Hands on experience in Apache Hadoop Map Reduce Programming and Pig Scripting.
  • Expert level understanding of Map Reduce internals, including shuffling, troubleshooting and partitioning. The bottlenecks in performance of a map reduce program.
  • Expertise in extending Hive and Pig functionalities by writing customized UDF’S.
  • Good knowledge on building Apache Spark applications.
  • Implemented Spark using Scala and SparkSQL for faster processing and testing of data.
  • Used Impala to read, write, and query the data in HDFS.
  • Experience with Oozie workflow Engine in running workflow jobs with actions that run Hadoop Map/Reduce and Pig jobs.
  • Experience in scripting for automation and monitoring using Python.
  • Excellent hands on experience with migrating data from different Relational Database Systems like MySQL and Oracle into HDFS and Hive and vice versa using Sqoop.
  • Good hands on experience in migrating unstructured data into HDFS using Flume/Kafka.
  • Imported data from Kafka consumer into HBase using spark streaming.
  • Good experience in NoSQL databases like HBase and MongoDB.
  • Involved in complete flow of the application, starting from data ingestion from upstream to HDFS, processing the data in HDFS and analyzing the data.
  • Involved in ETL, Data Integration, Ingestion and Migration.
  • Used Maven for building jar files of MapReduce programs and deployed to Cluster.
  • Generated various marketing reports using Tableau and Spotfire.
  • Implemented Continuous Integration using Jenkins.
  • Experience in Amazon AWS cloud services.
  • Configured HDFS to run on top of S3 to reduce the cost and leveraging the choice of a wide variety Amazon EC2 instances and Amazon EBS volumes for cluster's hardware, backup - and-restore to Amazon S3.
  • Good knowledge in Object Oriented Design Pattern concepts.
  • Expertise in Java Multithreading, Exception Handling, JSP, Servlets, JavaScript, jQuery, AJAX, CSS, HTML, Spring, Hibernate, Enterprise Java Beans, JDBC, RMI and XML related technologies.
  • Experience with version control systems like GitHub, SVN and CVS.
  • Good knowledge on SDLC, Agile and Waterfall Methodologies.
  • Involved in daily SCRUM meetings to discuss the development/progress and was active in making Scrum meetings more productive.
  • Excellent analytical, problem solving and interpersonal skills.
  • Ability to learn recent technologies and methodologies quickly. Consistent team player with excellent communication skills.

TECHNICAL SKILLS

Big Data Technologies: HDFS, YARN, Map Reduce, Pig, Hive, Sqoop, Flume, HBase, MongoDB, Oozie, Cassandra, Zookeeper, Spark, Kafka

Languages: C, C++, Java, J2EE, JavaScript, XML, PL/SQL, Python, Scala, Pig Latin, HiveQL

Frameworks: Spring, Hibernate, Struts

Web Technologies: HTML, CSS, Ajax, XML, jQuery

Web Servers: Apache Tomcat, WebSphere, WebLogic

Databases: Oracle, DB2, SQL Server, MySQL

IDE: Eclipse, NetBeans, SQL Server, MS Office Suite, Notepad++

Operating Systems: UNIX, LINUX, Windows, Mac, CentOS

PROFESSIONAL EXPERIENCE

Sr. Hadoop Developer

Confidential, Denver, CO

Responsibilities:

  • Wrote Map Reduce programs for data cleaning and preprocessing.
  • Involved in writing HiveQL to extract, transform and load the data into Database.
  • Wrote HiveQL queries and Pig Latin Scripts.
  • Wrote custom UDF's for Hive and Pig using Java.
  • Performed Hive partitioning, bucketing and executing different types of joins on Hive tables and implementing Hive Serves like JSON and Avro.
  • Fine tuning hive jobs for optimized performance.
  • Used Spark Streaming to divide streaming data into batches as an input to spark engine for batch processing.
  • Developed SparkSQL to load tables into HDFS to run select queries on top.
  • Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs.
  • Imported data from flume consumer into HBase using spark streaming.
  • Created Hive External tables and loaded data in to tables and queried data using HiveQL.
  • Experienced in using Zookeeper and Oozie Operational Services for coordinating the cluster and scheduling workflows.
  • Automated all the jobs, for ingesting data from relational databases to load data into Hive tables, using Oozie workflows.
  • Used Oozie workflow engine to manage interdependent Hadoop jobs and to automate several types of Hadoop jobs such as Java MapReduce, Hive and Sqoop as well as system specific job.
  • Experience in HBase database manipulation with structured, unstructured and semi-structured datasets.
  • Configured HDFS to run on top of S3 to reduce the cost and leveraging the choice of a wide variety Amazon EC2 instances and Amazon EBS volumes for cluster's hardware, backup-and-restore to Amazon S3.
  • Involved in ETL, Data Integration, Ingestion and Migration.
  • Implemented ETL pipelines to ingest data from traditional EDW Hadoop.
  • Generated various marketing reports using Tableau with Hadoop as a source for data.
  • Implemented Continuous Integration using Jenkins.
  • Involved in daily SCRUM meetings to discuss the development/progress and was active in making Scrum meetings more productive.

Environment: Apache Hadoop, Apache Hive, HDFS, Map Reduce, Pig, Spark, Oozie, Flume, Kafka, Tableau, Jenkins, Java, Python, Linux, Agile.

Hadoop Developer

Confidential, Atlanta, GA

Responsibilities:

  • Worked on Hive for exposing data for further analysis and for generating transforming files from different analytical formats to text files.
  • Expert level understanding of Map Reduce internals, including shuffling, troubleshooting and partitioning. The bottlenecks in performance of a map reduce program.
  • Developed Shell Scripts to perform Data Profiling on the ingested data with the help of Hive bucketing.
  • Involved in creating Hive tables for customer/Transaction, loading data into tables and partitioned tables using Hive Index and bucket to make ease data analytics.
  • Developed Pig Scripts, Pig and Hive UDFs to process the data.
  • Implemented Spark using Scala and SparkSQL for faster testing and processing of data.
  • Developed Map Reduce jobs for Log Analysis, Analytics and to generate reports for the number of activities created.
  • Used Impala to read, write, and query the data in HDFS.
  • Created HBase tables to load large data sets from UNIX, NoSQL and a variety of portfolios.
  • Used Flume to collect, aggregate, and store the web log data from various sources like web servers, network devices and pushed to HDFS.
  • Importing and exporting data into HDFS and Hive using Flume.
  • Used AWS services like EC2 and S3 for small data sets.
  • Created and involved in scheduling Oozie workflow engine to run Sqoop jobs.
  • Used Zookeeper and Oozie Operational Services for coordinating the cluster and scheduling workflows.
  • Implemented Continuous Integration using Jenkins.
  • Used Maven for building jar files of MapReduce programs and deployed to Cluster.
  • Used GitHub for version control across common source code used by developers.
  • Analyzing the issues and proactively identifying potential problems and their solutions.

Environment: Apache Hadoop, Hive, Map Reduce, HDFS, Pig, HBase, Cassandra, Oozie, Impala, AWS, Maven, GitHub, Jenkins, Linux.

Hadoop Developer

Confidential, Boca Raton, FL

Responsibilities:

  • Developing multiple MapReduce programs in java for data extraction, transformation and aggregation from multiple file formats including XML, JSON, CSV and other file formats.
  • Wrote Map Reduce code to process and parsing the data from various sources and storing parsed data into HBase and Hive using HBase-Hive Integration.
  • Experience in writing aggression logics in different combinations to perform complex data analytics to the business needs.
  • Experience in creating tables, dropping and altered at run time without blocking updates and queries using HBase and Hive.
  • Developed Java programs to process huge JSON files received from marketing team to convert into format standardized for the application.
  • Worked on different Big Data file formats like txt, sequence, Avro, Parquet and Snappy compression.
  • Involved in complete Big Data flow of the application starting from data ingestion from upstream to HDFS, processing the data in HDFS and analyzing the data.
  • Performed analysis of data using Pig scripting.
  • Implemented solutions for ingesting data from various sources and processing the data utilizing Big Data Technologies such as Hive, Pig, Sqoop, HBase, Map reduce, etc.
  • Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop.
  • Used Oozie workflow engine to manage interdependent Hadoop jobs and to automate several types of Hadoop jobs such as map-reduce Hive, Pig, and Sqoop.
  • Used Oozie to organize the MapReduce jobs and worked with HCatalog to open access to Hive's Metastore.

Environment: Apache Hadoop, Hive, Map Reduce, Pig, HBase, Sqoop, Oozie, Linux, Java, Agile.

Java Developer

Confidential

Responsibilities:

  • Involved in all the phases of SDLC process.
  • JDBC concept was used for database transaction management using create statements, prepared statements and callable statements to interact with the underlying MySQL Database.
  • Used various Core Java concepts such as Multi-Threading, Exception Handling, Collection APIs to implement various features and enhancements.
  • Application development using JSP, Servlets, JavaScript and JDBC Technologies.
  • Developed service layer logic for core modules using JSPs and Servlets.
  • Extensively used JavaScript to provide dynamic User Interface and for the client-side validations.
  • Developed the front-end screens using HTML, CSS, XML and JSP.
  • Used the Struts Validation Framework for validation and Struts Tiles Framework for reusable presentation components at the presentation tier.
  • Involved in using Hibernate for data persistence tier.
  • Worked on writing SQL Queries and PL/SQL Stored Procedures using Oracle. Used JDBC for connectivity.
  • Involved in creating software requirement specification document.
  • Implemented Maven as build and configuration tool.
  • Used Eclipse IDE for development.
  • Used GitHub for version control across common source code used by developers.

Environment: HTML, CSS, JSP, Oracle 10g, SQL, PL/SQL, Hibernate, Maven, GitHub, Eclipse, J2EE, CVS, Linux.

Java Developer

Confidential

Responsibilities:

  • The application is designed using J2EE design patterns and technologies based on MVC architecture.
  • Used HTML, jQuery, Java script and AJAX for implementing dynamic media play outs.
  • Extensively used various collections classes like Array List, Hash Map, Hash Table and Hash Set.
  • Generated Use case diagrams, Class diagrams, and Sequence diagrams using Rational Rose.
  • Involved in using Core java concepts - Collections, Exception Handling, Multi-Threading, Serialization, Thread pools, lock interface.
  • Used simple Struts Validation for validation of user input as per the business logic and initial data loading.
  • Implemented Struts Framework for configuration of action mappings and presentation logic in JSPs.
  • Developed Swing components such as JSlider, JButton and JCheckbox and AWT labels, textields and scrollbars.
  • Designed GUI screens using usability and design principles.
  • Multithreading is used to continuously keep track of downlink messages and send corresponding uplink messages.
  • Locks are used to synchronize resources sharing between these threads.
  • Writing database queries on Oracle 9i and Involved in the JDBC queries as part of implementation.
  • Involved in Unit Testing, User Acceptance Testing and Bug Fixing.
  • Used NetBeans IDE for development.
  • Involved in writing JUNIT test cases and Code version controlling using SVN.
  • Excellent written and verbal communication skills, inter-personal skills and self-learning attitude.

Environment: J2EE, JSP, NetBeans, Struts, HTML, Java Script AJAX, Oracle 9i, Swing, Linux, SVN.

We'd love your feedback!