We provide IT Staff Augmentation Services!

Sr.hadoop Developer Resume

0/5 (Submit Your Rating)

Oak Brook, IL

SUMMARY

  • Over 10+years of total experience in software development life cycle design, development and support of systems application architecture.
  • 4 years of experience in Big Data Hadoop ecosystem experience in ingestion, storage, querying, processing and analysis of big data.
  • Proficiency and knowledge in understanding the architectural components ofHadooplike HDFS, JobTracker, TaskTracker, NameNode,DataNode and MapReduce concepts.
  • Hands on experience in developing and deploying enterprise based applications using major components inHadoopecosystem likeHadoopMap Reduce, YARN, Hive, Pig, HBase, Flume, Sqoop, Spark (Streaming, Spark SQL), Storm, Kafka, Oozieand, ZooKeeper.
  • Hand on experience of multiple distributions Cloudera, Hortonworks.
  • Hands on experience in designing and creating Hive tables using shared meta - store with partitioning and bucketing.
  • Hands on experience in creating custom UDFs functions including UDAFs & UDTFs in Hive & Pig.
  • Hands on experience in using Pig for analysis on large unstructured data sets.
  • Experience in developingHadoopMapReduce applications using Java andHadoopAPI's.
  • Automated Sqoop, Hive and Pig scripts using work flow scheduler Oozie, Spark and Kafka
  • Experience in developing and designing POCs using Scala and deployed on the Yarn cluster, compared the performance of Spark, with Hive and SQL/Teradata.
  • Hands on experience in converting Hive/SQL queries into Spark transformations using Scala.
  • Hands on experience in writing spark streaming application.
  • Hands on experience with NoSQL Databases like MongoDB, HBase and Cassandra.
  • Rich experience working with Shell Scripting.
  • Hands on experience in using Java Technologies including J2EE design Patterns like Singleton, Factory, MVC, Front Controller, for reusing most effective and efficient strategies.
  • Hands on Experience in using different applications development frameworks like Hibernate, Struts, and spring for developing integrated applications and different light weight business components.
  • Hands on Experience in developing service components using JDBC.
  • Experience in developing and designing Web Services (SOAP and Restful Web services)
  • Extensive experience in writing SQL queries for Oracle,Hadoopand DB2 databases using SQLPLUS.
  • Hands on experience in working with Oracle, DB2, MySQL and knowledge on SQLServer.
  • Hands on experience in using SQL and PL/SQL to write Stored Procedures, Functions and Triggers.
  • Ability to adapt to evolving technology, strong sense of responsibility and accomplishment.

TECHNICAL SKILLS

Hadoop Technologies: HDFS, MapReduce, Hive, Impala, Pig, Sqoop, Flume, Oozie, Zookeeper, Ambari, Hue, Spark, Strom, Talend, Ganglia

Operating System: Windows, Unix, Linux

Languages: Java, J2EE, SQL, PL/SQL, Shell Script

Project Management / Tools: MS Project, MS Office, TFS, HP Quality Center Tool

ETL Technologies: Microsoft SQL Server Integration Services (SSIS 2008 Microsoft SQL Server Integration Services (SSIS 2008)

Front - End: HTML, JSTL, DHTML, JavaScript, CSS, XML, XSL, XSLT

Databases: MySQL, Oracle 11g, SQL ServerNoSQL Databases HBase, Cassandra,File System HDFS

Reporting Tools: Jasper Reports, Tableau

IDE Tools: Eclipse, NetBeans

Application Server: Apache Tomcat, Web Logic

PROFESSIONAL EXPERIENCE

Sr.Hadoop Developer

Confidential, Oak Brook, IL

Responsibilities:

  • Responsible for building scalable distributed data solutions usingHadoop.
  • Written multiple Map Reduce programs in Java for Data Analysis.
  • Wrote Map Reduce job using Pig Latin and Java API.
  • Performed performance tuning and troubleshooting of Map Reduce jobs by analyzing and reviewingHadooplog files.
  • End-to-end performance tuning ofHadoopclusters andHadoopMapReduce routines against very large data sets.
  • Involved in loading data from LINUX file system to HDFS.
  • Developed hive queries and UDFS to analyze/transform the data in HDFS.
  • Importing and exporting data into HDFS and Hive using Sqoop and Flume.
  • Developed Pig scripts for analyzing large data sets in the HDFS.
  • Collected the logs from the physical machines and the OpenStack controller and integrated into HDFS using Flume.
  • Designed and presented plan for POC on Impala.
  • Knowledge on handling Hive queries using Spark SQL that integrates with Spark environment.
  • Implemented Avro and parquet data formats for Apache Hive computations to handle custom business requirements.
  • Responsible for creating Hive tables, loading the structured data resulted from MapReduce jobs into the tables and writing Hive queries to further analyze the logs to identify issues and behavioral patterns.
  • Worked on Sequence files, RC files, Map side joins, bucketing, partitioning for Hive performance enhancement and storage improvement.
  • Implemented Daily Cron jobs that automate parallel tasks of loading the data into HDFS using Oozie coordinator jobs.
  • Responsible for performing extensive data validation using Hive
  • Sqoop jobs, PIG and Hive scripts were created for data ingestion from relational databases to compare with historical data.
  • Involved in loading data from Teradata database into HDFS using Sqoop queries.
  • Involved in submitting and tracking MapReduce jobs using Job Tracker.
  • Involved in creating Oozie workflow and Coordinator jobs to kick off the jobs on time for data availability.
  • Used Pig as ETL tool to do transformations, event joins, filter and some pre-aggregations
  • Used Visualization tools such as Power view for excel, Tableau for visualizing and generating reports.
  • Exported data to Tableau and excel with Power view for presentation and refining
  • Implemented business logic by writing Pig UDFs in Java and used various UDFs from Piggybanks and other sources
  • Implemented Hive Generic UDF's to implement business logic.
  • Implemented test scripts to support test driven development and continuous integration.
  • Involved in story-driven agile development methodology and actively participated in daily scrum meetings.

Environment: ApacheHadoop, Map Reduce, HDFS, Pig, Hive, Sqoop, Flume, Oozie, Java, C++, Linux, Maven, Teradata, Zookeeper, Tableau.

Sr. Hadoop Developer

Confidential, New York, NY

Responsibilities:

  • Expertise in designing and deployment ofHadoopcluster and different Big Data analytic tools including Pig, Hive, HBase, Oozie, ZooKeeper, SQOOP, flume, Kafka, Spark, Cassandra with Hortonworks and Cloudera
  • InstalledHadoop, Map Reduce, HDFS, AWS and developed multiple MapReduce jobs in PIG and Hive for data cleaning and pre-processing.
  • Understanding business needs, analysing functional specifications and map those to develop and designing MapReduce programs and algorithms.
  • Written Pig and Hive jobs to parse the logs and structure them in tabular format to facilitate effective querying on the log data. Also have hand on Experience on Pig and Hive User Define Functions (UFD).
  • Execution ofHadoopecosystem and Applications through Apache HUE.
  • OptimizingHadoopMapReduce code, Hive/Pig scripts for better scalability, reliability and performance.
  • Developed the OOZIE workflows for the Application execution.
  • Feasibility Analysis (For the deliverables) - Evaluating the feasibility of the requirements against complexity and time lines.
  • Performing data migration from Legacy Databases RDBMS to HDFS using SQOOP.
  • Writing Pig scripts for data processing.
  • Implemented Hive tables and HQL Queries for the reports. Written and used complex data type in Hive. Storing and retrieved data using HQL in Hive. Developed Hive queries to analyze reducer output data.
  • Highly involved in designing the next generation data architecture for the unstructured data
  • Managed a 4-nodeHadoopcluster for a client conducting aHadoopproof of concept. The cluster had 12 cores and 3 TB of installed storage.
  • Developed PIG Latin scripts to extract data from source system.
  • Involved in Extracting, loading Data from Hive to Load an RDBMS using SQOOP.
  • Integrate four square monitoring and production system with Kafka
  • Designed, documented operational problems by following standards and procedures using a software reporting tool JIRA.
  • Worked on Data modeling during application software design

Environment: HDFS, Map Reduce, Hive, Oozie, Java, PIG, Shell Scripting, Kafka, Linux, HUE, SQOOP, Flume, DB2, and Oracle 11g, Data modeling

Sr.Hadoop Developer

Confidential, Dublin, OH

Responsibilities:

  • Designed and implemented Bigdata, Data Warehouse using HBase, Hive, and Pig.
  • Designed and developed ETL processes to load data from relational database systems into the HDFS using various Sqoop.
  • Demonstrated strong expertise on Big Data modeling with Hive and Hbase.
  • Demonstrated strong expertise on using Sqoop, Oozie, Hive, and Pig for ETL process.
  • Demonstrated strong expertise on writing PIG UDFs, Map Reduce jobs with different file format suchas XML, CSV, JSON data.
  • Good understanding and hands on experience in predictive modeling.
  • Led the team in making valuable decisions by performing analytical operations.
  • Predicted consumer behavior, such as what products a particular user has bought and made predictions/recommendations based on recognizing patterns by using Hadoop, Hive and Pig queries.
  • Installed and configured Hadoop, Map Reduce, and HDFS.
  • Developed multiple Map Reduce jobs using Java API for data cleaning and pre-processing.
  • Importing and exporting data into HDFS and HIVE from Oracle 11g database using Sqoop.
  • Responsible to manage data coming from different sources.
  • Trouble shooting cluster issues and various ecosystem component issues.
  • Monitoring the running Map Reduce programs on the cluster.
  • Responsible for loading data from UNIX file systems into HDFS.
  • Installed and configured Hive.
  • Involved in creating Hive Tables, loading with data and writing Hive queries which will invoke and run Map Reduce jobs in the back end.
  • Installed and configured Pig.
  • Wrote Pig scripts to process unstructured data and create structure data for use with Hive.
  • Developed scripts and automated data management from end to end and sync up b/w all the clusters.

Environment: Agile SCRUM, Hadoop (HDFS/Map Reduce), Pig, Hive, UDF, Sqoop, Oozie, Cloudera, Linux, Windows 7.

Sr. Java Developer

Confidential, Dallas, TX

Responsibilities:

  • Worked with sprint planning, sprint demo, status and daily standup meeting.
  • Developed the application using Spring Web MVC framework.
  • Worked with Spring Configuration files to add new content to the website.
  • Worked on the Spring DAO module and ORM using Hibernate. Used Hibernate Template and Hibernate Dao Support for Spring-Hibernate Communication.
  • Extensively used spring’s features such as Dependency Injection/Inversion of Control to allow loose coupling between business classes (POJOs)
  • Used Hibernate framework to retrieve and update information and dependency Injection is achieved by Spring MVC Framework.
  • Configured Association Mappings such as one-one and one-many in Hibernate
  • Worked with JavaScript calls as the Search is triggered through JS calls when a Search key is entered in the Search window
  • Refined feature requirements and developed user interface components using Oracle ADF and JSF.
  • Worked on ADF Model and View Controller projects.
  • Designed and developed the interface for data search using ADF Faces.
  • Designed tabbed ADF pages using ADF dynamic tab shell template.
  • Worked on analyzing other Search engines to make use of best practices.
  • Collaborated with the Business team to fix defects.
  • Worked on XML, XSL and XHTML files.
  • Interacted with project management to understand, learn and to perform analysis of the Search Techniques.
  • As part of the team to develop and maintain an advanced search engine, would be able to attain expertise on a variety of new software technologies.

Environment: Java 1.6, J2EE, Eclipse SDK 3.3.2, Java Spring 3.x, JQuery, Oracle 10i, Hibernate, JPA, JSON,Apache Ivy, SQL, stored procedures, Shell Scripting, JQuery, XML, HTML and JUnit, TFS, Ant, Visual StudioPremium 2010, Rational Clear quest.

Java Developer

Confidential

Responsibilities:

  • Involved in full development life cycle of the project from analysis, designing and testing.
  • Involved in preparing class diagrams, use diagrams and sequence diagrams.
  • Worked on the User Interface using JSP, JavaScript and HTML.
  • Developed the Test cases for unit testing.
  • Involved in gathering requirements, design and implemented the application using Java/J2EE technologies.
  • Created Static and dynamic web pages, JSP and Servlets.
  • Implemented presentation tier on Servlets, JSP, and DHTML for custom business requirements.
  • Played an active role in testing the application by writing test cases for different scenarios and extensively involved in bug fixing.
  • Involved in deploying the application in different environments.
  • Gathered business requirements from the end users and wrote functional specifications and design documents.
  • Worked with onsite team in coordinating the work.
  • Used JavaScript for Client Side validations.
  • Wrote backend-stored procedures, functions and triggers.
  • Deployed the Application into UNIX Server after the development and testing was finished.
  • Handled the issues that came during the Production time and resolved them.
  • Maintained Mail Communication between the users to update them regarding process enhancement.

Environment: Java/J2EE, JSP, HTML, PL/SQL Oracle, UNIX, JavaScript, and Servlets.

We'd love your feedback!