SUMMARY

• Certified Hadoop Developer with 10+ years of extensive experience in IT including around 5 years of Hands on experience in Big Data Ecosystem components.
• Experienced in involving complete SDLC life cycle includes requirement gathering, design, development, testing and production deployment.
• Excellent knowledge/understanding of Hadoop architecture like HDFS (Hadoop Distributed File System), Sqoop, Hive, HBase, Spark, Hue/Ambari, MapReduce framework, Kafka, Yarn, Oozie, Zookeeper.
• Worked in Big Data Hadoop Distributed distributions like Cloudera & Hortonworks.
• Good understanding of core Java, Eclipse, UNIX shell scripting, Linux, IDE.
• Hands on experience in writing MapReduce programs using java to handle different types of datasets using map and reduce tasks.
• Strong experience in architecting real time streaming applications and batch style large scale distributed computing applications using tools like Spark Streaming, Spark SQL, Kafka, Flume, MapReduce, Hive etc.,
• Experience in developing a data pipeline to store data into HDFS and importing the real time data to Hadoop environment using Kafka.
• Orchestrated various Sqoop, Hive, Spark scripts using Oozie workflows and sub workflows by configuring coordinators and workflow XMLs.
• Strong experience in analyzing large amounts of datasets writing Hive queries.
• Extensive experience in working with structured data using Hive QL, join operations, writing custom UDFs and optimizing Hive queries.
• Responsible for handling different file formats like Avro, Parquet, ORC, Text formats.
• Experience in import/export of data from/to Hadoop & RDBMS using Sqoop scripts.
• Good hands on experience in converting MapReduce programs into Spark RDD transformations and actions improve performance.
• Experienced in working with different scripting technologies like Python, Unix Shell Scripts.
• Skilled at build/deploy multi module applications using Maven.
• Strong analytical, trouble shooting and debugging ability with excellent understanding of frameworks.
• Adequate knowledge and working experience in Agile and Waterfall methodologies.
• Great team player, quick learner with effective communication, being an SME conducted several knowledge transfer sessions to mentees, exhibits leadership skills.

TECHNICAL SKILLS

Operating System: Windows, Unix, Linux

Programming Languages: Java, Python, SQL

Big Data Ecosystem: HDFS, Map Reduce, Hive, Flume, Sqoop, Apache Spark, Spark SQL, Spark Streaming, Kafka, HBase, Zookeeper, and Oozie.

Hadoop Distributions: Cloudera, Hortonworks

DB Languages: MySQL, Oracle, Teradata, DB2, Hive, SqlServer

Scripting Languages: Shell scripting, Python

Tools: Eclipse, Maven, Accurev, WinscP, Ambari, Hue, Kerberos, Putty, Alteryx

PROFESSIONAL EXPERIENCE

Confidential, Dearborn Michigan

Senior Hadoop Developer

Responsibilities:

• Involved in complete SDLC of project includes requirements gathering, design documents, development, testing and production environments.
• Worked collaboratively with all levels of business stakeholders to design, implement and test Big Data based analytical solution from various sources.
• Developed several data entities through HQLs to transform the data for business need.
• Developed optimal strategies for distributing the web log data over the cluster; importing and exporting the stored web log data into HDFS and Hive using Sqoop.
• Collected and aggregated large amounts of data from different DSC (Data Supply Chain), transform them based on business needs and stored the data into HDFS/Hive for analysis.
• Implemented Hive Generic UDF's to incorporate business logic into Hive Queries.
• Converted Hive queries into Spark SQL for optimization and ensure data availability to business customers.
• Involved in landing source data (Mainframe, DB2, SqlServer, Teradata) into Hadoop environment.
• Integrated Oozie with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (such as Map - Reduce, Pig, Hive, and Sqoop) as well as system specific jobs (such as Java programs and shell scripts)
• Collected and aggregated large amounts of web log data from different sources such as webservers, mobile and network devices using Apache Kafka and stored the data into HDFS for analysis.
• Monitored the error logs using yarn logs, debugging the code serially and fixed the problems.
• Involved in Agile methodologies, daily stand up meetings, PI planning in PDO model.
• As SME of the applications and processes, participate in the design review to give insights.
• Panel member for code review council, and provide valuable review comments to peers.

Technologies/Tools used: Java/J2EE, Eclipse, Maven, SQL, Apache Hadoop, Map-Reduce, Hive, Sqoop, Oozie, Apache Spark, Spark SQL, HBase, SqlServer, Teradata, Linux, XML, WinSCP, Accurev, Putty

Confidential, Dearborn Michigan

Hadoop Developer

Responsibilities:

• Involved in creating Hive tables and loading and analyzing data using hive queries.
• Conducted business requirement meetings with Business Analyst and Business to comprehend the requirements and freeze them.
• Developed Simple to complex Map Reduce Jobs using Hive.
• Extensively worked on improving the performance and HQL optimization of existing processes in Data ingestion and on reconciliation.
• Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Hive.
• Migrated complex map reduce programs into in memory Spark processing using Transformations and actions.
• Exported the analyzed data to relational databases using Sqoop for visualization and to generate reports for the BI team
• Involved in complete PROD support- Deployments, reviews, source team coordination, developing and scheduling additional jobs.
• Developed the Sqoop scripts in order to import data from RDBMS to HIVE, RDBMS to HDFS and Export Data from HDFS to RDBMS.
• Notifications to downstream if the team would not be able to provision the data to various downstream applications.
• Mentored analyst and test team for writing Hive Queries.
• Supported Testing teams on writing test cases and executing the same during Functional Testing.

Technologies/Tools used: Oracle, Java/J2EE, Map-Reduce, Pig, Hive, Sqoop, Oozie, Spark, Teradata, Maven, Shell scripting

Confidential, Riverwoods Illinois

Hadoop Data Engineer

Responsibilities:

• Involved in initial meetings with business to understand the requirements.
• Involved in complete SDLC of project includes requirements gathering, design documents, development, testing and production environments.
• Developed optimal strategies for distributing the web log data over the cluster; importing and exporting the stored web log data into HDFS and Hive using Sqoop.
• Implemented Hive Generic UDF's to incorporate business logic into Hive Queries.
• Integrated Oozie with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (such as Map-Reduce, Pig, Hive, and Sqoop) as well as system specific jobs.
• Involved in Agile methodologies, daily scrum meetings, spring planning.
• Involved in bringing the data from DB2 to HIVE environment and scrubbing process for all release environments and implemented successfully.
• Presented deck on the same to client team and it was well received by them.
• Involved in creating Hive tables, and loading and analyzing data using hive queries
• Mentored analyst and test team for writing Hive Queries.
• Experienced in running Hadoop streaming jobs to process terabytes of xml format data.
• Load and transform large sets of structured, semi structured and unstructured data.
• Responsible to manage data coming from different sources.
• Assisted in exporting analyzed data to relational databases using Sqoop.

Technologies/Tools used: Java/J2EE, Map-Reduce, Pig, Oracle, Teradata, MySQL, Hive, Sqoop, Oozie, Maven, Spark

Confidential, Milwaukee Wisconsin

Mainframe Senior Developer

Responsibilities:

• User Manual, Release Notes and Help Manual Design for Sales teams and customers
• Provide Status Report on weekly and Monthly basis
• Taking Ownership for the Product on the production move from QA front without issue.
• Requirement gathering from business on adding ACB page to the existing IFLs.
• Scheduling meeting with business for understanding the requirement even better.
• Creating design document on adding new ACB page to the existing ledgers and present it in design council for approval.
• Perform impact analysis on the code changes and interact with other impacted teams to make sure the code changes would not affect their system and involve them for downstream validation.
• Coding the application program to add ACB pages to all type of policies(90L,65L,ACL,ECL)
• Test the code to produce all type of illustrations like PDR, IFL, NAIC basics reports to make sure it has added ACB page. Also need to test the code with various IFRP systems such as NIS, Executive Benefit, and Client file system.
• Arrange and conduct review, bug triage and Sprint retrospective meetings
• Deploy the code in production and hand over the installed piece to application support team.
• Planning for the Demo Session with the End User, for providing /taking suggestion for the Qualitative product deliverable
• Taking Ownership for the Product on the production move

Technologies/Tools used: ZOS, COPYBOOK, COBOL, VSAM, JCL, CICIS, File Aid, SPUFI, CA7, IMS, CHANGEMAN, DB2

Confidential

Mainframe Developer

Responsibilities:

• Demonstrated track record of deploying business process requirements into production environment with needed approvals from business and client team and playing the role of functional and technical lead.
• Developed new programs using COBOL, DB2, IMS, CICS.
• Created test cases and performed Unit / System / Performance / Integration Testing to improve the performance of DB2 Stored Procedures.
• Executed DB2 jobs to bind plans and packages related to DB2 stored procedures.
• Provide on-call support for production job abends and fix them immediately to avoid SLA violation and later develop the permanent fix for the technical Job abends.
• Performed many CICS operations during testing phase: Bringing the CICS region up and down, CEDA for defining a new transaction, CEDF for debugging the CICS programs, CEMT to make new copy of CICS programs, open and close VSAM files.
• Worked with DB2 support to set up Tables, Table spaces, Indexes, Stored procedures etc. in the new regions and provided support during various DB2 upgrades.
• Implemented many fixes in production after representing with change council that benefitted the team to be always green in change chart. No back outs due to bad changes.

Technologies/Tools used: ZOS, Mainframe, COBOL, VSAM, JCL, CICIS, File Aid, SPUFI, CA7, IMS, LIBRARIAN, DB2