We provide IT Staff Augmentation Services!

Hadoop Developer Resume

3.00/5 (Submit Your Rating)

Raleigh North, CarolinA

SUMMARY

  • 7+ years of IT Experience in Architecture, Analysis, design, development, implementation, maintenance and support, with experience in developing strategic methods for deploying big data technologies to efficiently solve Big Data processing requirements
  • Experience in Data warehouse, Test and Production Environments on various business domains like Financial, Insurance and Banking
  • 3 years of experience on BIG DATA using HADOOP and SPARK framework and related technologies such as HDFS, HBASE, MapReduce, HIVE, PIG, FLUME, OOZIE, SQOOP, and ZOOKEEPER.
  • Experience in data analysis using HIVE, Pig Latin, HBase and custom Map Reduce programs in Java.
  • Experience in writing custom UDFs in java for Hive and Pig to extend the functionality.
  • Experience in writing MAPREDUCE programs in java for data cleansing and preprocessing.
  • Excellent understanding/knowledge on Hadoop and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, Resource Manager (YARN).
  • Experience in working with Flume to load the log data from multiple sources directly in to HDFS.
  • Worked with different file formats like flat files, Sequence, Avro and Parquet.
  • Experience with compressing data with different algorithms like gzip, bzip.
  • Well versed with Schema design and Performance tuning.
  • Excellent understanding and knowledge of NOSQL databases like HBase.
  • Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems (RDBMS) and from RDBMS to HDFS.
  • Built real - time Big Data solutions using HBASE handling billions of records.
  • Good experience working with MapR and Cloudera Distribution.
  • Experience in designing both time driven and data driven automated workflows using Oozie.
  • Experience working with Apache SOLR.
  • Experience in Object Oriented Analysis Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns and Core Java design patterns.
  • Experience working with JAVA, J2EE, JDBC, ODBC, JSP, Java Eclipse, Java Beans, EJB, Servlets, MS SQL Server.
  • Extensive experience working in Oracle, DB2, SQL Server and My SQL database.
  • Experience in writing UNIX shell scripts and Python scripts.
  • Experience in all stages of SDLC (Agile, Waterfall)
  • Experience writing Technical Design document, Development, Testing and Implementation of Enterprise level Data mart and Data warehouses.
  • Good team player, strong interpersonal and communication skills combined with self-motivation, initiative and the ability to think outside the box.

TECHNICAL SKILLS

Hadoop Ecosystem: MapReduce, HDFS, Hive, Pig, Sqoop Zookeeper, Oozie, Flume and HBase

Language: C, C++, Java, J2EE, Python, UML

Web Technologies: JavaScript, JSP, Servlets, JDBC, Unix/Linux Shell Scripting, Python, HTML, XML

Methodologies: Waterfall, Agile/Scrum.

Databases: Oracle, MySQL, HBase

Application/Web server: Apache Tomcat, WebSphere and JBoss.

IDE’s: Eclipse, Netbeans

ETL & Reporting Tools: Informatica, SAP Business Objects, Tableau

PROFESSIONAL EXPERIENCE

Confidential, Raleigh, North Carolina

Hadoop Developer

Responsibilities:

  • Worked on Algorithm Development for analyzing click stream data using Spark and Scala
  • Database migrations from Traditional Data Warehouses to Spark Clusters.
  • Developed Spark SQL Scripts for data Ingestion from Oracle to Spark Clusters and relevant data joins.
  • Data Workflows and Pipelines are created for transition and analyzing trends using Spark Mllib.
  • Entire Project is set up on Amazon Web Services Cloud.

Environment: Cloudera Hadoop Framework, AWS, Apache Spark, Scala, Spark Mllib, Spark SQL, PostgreSQL, Hive

Confidential, Des Moines, IA

Hadoop Developer

Responsibilities:

  • Written the Apache PIG scripts to process the HDFS data.
  • Created Hive tables to store the processed results in a tabular format.
  • Developed the sqoop scripts in order to make the interaction between Pig and MySQL Database.
  • Involved in gathering the requirements, designing, development and testing
  • Writing the script files for processing data and loading to HDFS
  • Storing and retrieved data using HQL in Hive.
  • Developed the UNIX shell scripts for creating the reports from Hive data.
  • Data Ingestion (Flume, Sqoop, learning Kafka), Data pipeline architecture, Data cleansing, ETL, Processing and some visualization experience. Enable CDH to consume data from customer’s enterprise tool (I have worked with sources like RabbitMQ, IBM MQ, RDBMS, etc)
  • Use-case development (Hive, Pig, Spark, Spark Streaming); Implemented MapReduce to discover interesting patterns in data.
  • Providing documentation on the architecture, deployment and all details the customer would require to run the CDH cluster as part of a “delivery document(s)”
  • RDBMS: MySQL and Postgresql (some experience to support it as a backend for Hive Metastore, Cloudera Manager related components, Oozie etc.)
  • Provide Subject Matter Expertise on Linux (To support running CDH/Hadoop optimally on the underlying OS).
  • Training customers/partners when required.
  • Understanding customer requirements and identifying how the Hadoop eco-system could be leveraged to implement their requirements into Hadoop, how CDH can fit into their current infrastructure, where Hadoop can complement existing products, etc.

Environment: Cloudera Hadoop Framework, MapReduce, Hive, Pig, HBase, Business Objects, Platfora, HParser, Java, Python, UNIX Shell Scripting.

Confidential, Omaha, Nebraska

Hadoop Developer

Responsibilities:

  • Worked with the business users to gather, define business requirements and analyze the possible technical solutions.
  • Developed job flows in Oozie to automate the workflow for Pig and Hive jobs.
  • Designed and built the reporting application that uses the Spark SQL to fetch and generate reports on HBase table data.
  • Extracted feeds from social media sites such as Facebook, Twitter using Python scripts.
  • Implemented helper classes that access HBase directly from Java using Java API.
  • Integrated MapReduce with HBase to import bulk amount of data into HBase using MapReduce programs.
  • Experienced in converting ETL operations to Hadoop system using Pig Latin Operations, transformations and functions.
  • Extracted the needed data from server and into HDFS and bulk loaded the cleaned data into HBase.
  • Handled different time series data using HBase to store data and perform analytics based on time to improve queries retrieval time.
  • Participated with admins in installation and configuring Map Reduce, Hive and HDFS.
  • Implemented CDH3 Hadoop cluster on CentOS, assisted with performance tuning and monitoring.
  • Used Hive to analyze data ingested into HBase and compute various metrics for reporting on the dashboard.
  • Managed and reviewed Hadoop log files.
  • Involved in review of functional and non-functional requirements.

Environment: Hadoop Framework, MapReduce, Hive, Sqoop, Pig, HBase, Flume, Oozie, Java, Python, UNIX Shell Scripting, Spark.

Confidential, Des Moines, IA

Hadoop Developer

Responsibilities:

  • Analyzed large data sets by running Hive queries and Pig scripts
  • Worked with the Data Science team to gather requirements for various data mining projects
  • Involved in creating Hive tables, and loading and analyzing data using hive queries
  • Developed Simple to complex MapReduce Jobs using Hive and Pig
  • Involved in running Hadoop jobs for processing millions of records of text data
  • Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required
  • Developed multiple MapReduce jobs in java for data cleaning and preprocessing
  • Involved in loading data from LINUX file system to HDFS
  • Responsible for managing data from multiple sources
  • Responsible to manage data coming from different sources
  • Assisted in exporting analyzed data to relational databases (mysql) using Sqoop
  • Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts
  • Generating tableau reports and building dashboards
  • Worked closely with business units to define development estimates according to Agile Methodology

Environment: JMS, Sonic Management, Apache Hadoop, Hbase, Hive, Oozie, Crunch, Map/Reduce, Pig, Hive, Java, SQL.

Confidential

Java/J2EE Developer

Responsibilities:

  • Developed Servlets and JSP based on MVC pattern using Struts Action framework.
  • Parsing high-level design spec to simple ETL coding and mapping standards
  • Involved in writing Hibernate queries and Hibernate specific configuration and mapping files.
  • Used Log4J logging framework to write Log messages with various levels.
  • Involved in fixing bugs and minor enhancements for the front-end modules.
  • Used JUnit framework for writing Test Classes.
  • Coded various classes for Business Logic Implementation.
  • Preparing and executing Unit test cases
  • Doing functional and technical reviews
  • Assuring quality in the deliverables.
  • Conducted Design reviews and Technical reviews with other project stakeholders.
  • Implemented Services using Core Java.
  • Developed and deployed UI layer logics of sites using JSP.
  • Used built-in/custom Interceptors and Validators of Struts.
  • Involved in the complete life cycle of the project from the requirements to the production support.

Environment: J2EE, JDBC, Java, Servlets, JSP, Struts, Hibernate, Web services, SOAP, WSDL, Design Patterns, MVC, HTML, JavaScript 1.2, WebLogic, XML and JUnit.

Confidential

Java Developer

Responsibilities:

  • Developed User Interfaces module usingJSP,JavaScript, DHTML and form beansfor presentation layer.
  • Developed Servlets and Java Server Pages (JSP).
  • Developed PL/SQL queries, and wrote stored procedures andJDBC routines to generate reports based on client requirements.
  • Enhancement of the System according to the customer requirements.
  • Involved in the customization of the available functionalities of the software for an NBFC (Non-BankingFinancialCompany).
  • Involved in putting proper review processes and documentation for functionality development.
  • Providing support and guidance for Production and Implementation Issues.
  • Used Java Script validation in JSP.
  • UsedHibernateframework to access the data from back-end SQL Server database.
  • Used AJAX (Asynchronous JavaScript and XML) to implement user friendly andefficient client interface.
  • UsedMDBfor consuming messages from JMS queue/topic.
  • Designed and developed Web Application usingStrutsFramework.
  • ANT to compile and generate EAR, WAR, and JAR files.
  • Created test case scenarios for Functional Testing and wrote Unit test cases with JUnit.
  • Responsible for Integration, unittesting, systemtesting and stress testing for all the phases of project.

Environment: Java SE 6, Servlets, XML, HTML, JavaScript, JSP, Hibernate, Oracle 11g, SQL Navigator.

We'd love your feedback!