PROFESSIONAL SUMMARY

• Over 12 years of IT experience in various technologies that includes Hadoop/Big Data, ETL, Data Warehouse, and Databases.
• Have worked for clients from Financial Services, Retail and Telecom as Senior Big Data Consultant and Developer. Successfully executed various projects under various platforms.
• A quick learner who can absorb new ideas and can communicate clearly and effectively.
• Attention to detail & ability to work in a pressurized time sensitive environment.
• Worked with the customers to develop a vision for Big Data and Analytics.
• Implemented all the projects in accordance with the client’s best practices with standard SDLC / Agile methodologies, procedures and techniques.
• Effective Team Player with excellent analytical, programming, written and verbal communication skills.

TECHNICAL BACKGROUND:

• Experienced in AWS EC2, S3, EMR, Lambda, Data pipeline, SNS, Boto3, and Redshift.
• Experienced in writing Spark Applications using Python.
• Experienced in writing Hive, Spark SQL scripts.
• Experienced in analyzing raw data with the help of BigData Technology Stack.
• Experienced in Java writing Map Reduce Programs.
• Experienced in ETL Tools (Ab>Initio, Informatica, Talend).
• Experienced running complex and high volume ETL processes using Hive.
• Experienced in Cloudera, Hortonworks environments.
• Good Knowledge on Hadoop Cluster YARN architecture.
• Highly skilled in Design, Development, Implementation of database applications on UAT, Staging and Production databases.

PROFESSIONAL EXPERIENCE

Confidential

Sr. Bigdata Consultant

Environment: Spark, Python, Airflow, Hive, Shell scripting, AWS EMR, Redshift.

Responsibilities:

• Developed data pipelines using spark and hive for Campaign team that loads the response generated from the emails and app activity of the customers of Confidential to analyze the trends and effectiveness of the campaigns that have sent out to the customers.
• Source date, curate and syndicate KPI s from multiple consumer touchpoints ranging from email campaigns, mobile fitness apps (NRC + NTC), online and mobile shopping experiences ( Confidential .com, SNKRS and Confidential App) and brick and mortar consumer offerings.
• Coordinated with Data science team to come up with new business requirements and implemented them on the Data pipelines.
• Worked on Airflow workflows for scheduling the hive and spark jobs. participated in design solution brain storm sessions to create standard framework to be developed to capture the metrics of the spark loads to redshift and do the data quality check and subsequently start or stop the subsequent jobs in order to maintain the data correctness and reducing the customer complaints.
• Defined standards, best practices and design patterns to make the applications efficient, scalable and easily maintainable.
• Built scripts and utilities for automating and streamlining many processes and written optimized HQL s jobs.
• Expert in analyzing complex problems and identifying root cause and providing solutions

Confidential

Sr. Bigdata Consultant

Environment: & Responsibilities: Cloudera, Hive, Spark, AWS S3, EMR, Lambda, Datapipeline.

Responsibilities:

• Converted lot of existing ETL loads in the on premise cluster to AWS Cloud.
• Our typical ETL load starts with an event generated from the ingestion team bucket calling the lambda function to bring up the cluster then kick of the data load through data pipeline.
• Coordinated with Decision science team to come up with new business requirements and implemented them on the Data pipelines.
• Generated Revenue per page reports through an automated solution using Hive and Shell Scripts replacing the reporting layer.
• Coordinated with Decision science team to come up with business requirements and implemented them in Hive.
• Worked on Control - M workflows for scheduling the hive and spark queries. participated in design solution brain storm sessions to create standard framework to be developed to capture the metrics of the hive loads and do the data quality check and subsequently start or stop the subsequent jobs in order to maintain the data correctness.
• Defined standards, best practices and design patterns to make the applications efficient, scalable and easily maintainable.
• Built scripts and utilities for automating and streamlining many processes and written optimized HQL s jobs.
• Expert in analyzing complex problems and identifying root cause and providing solutions

Confidential

Hadoop Consultant

Responsibilities:

• Generated User Activity Aggregate for Business Users to analyze the iTunes users listening patterns on the songs with some statistics.
• Consolidated business requirements and implemented them in Hive.
• Worked on Oozie workflows for scheduling the hive queries.
• Defined standards, best practices and design patterns to make the applications efficient, scalable and easily maintainable.
• Built UDF s and scripts and utilities for automating and streamlining many processes and written optimized HQL s jobs.
• Expert in analyzing complex problems and identifying root cause and providing solutions

Confidential

Hadoop / Ab>Initio Consultant

Environment: & Responsibilities: Cloudera CDH 5.3.2, YARN, Pig, Ab>Initio.

Responsibilities:

• Generated EDW detail data in Hadoop based on Confidential Global and Geo Enriched files.
• Transformed some AbInitio Graphs into Pig Scripts.
• Worked on some graphs where the ETL logic to be enhanced and tweaked to the new requirement.
• Defined standards, best practices and design patterns to make the applications efficient, scalable and easily maintainable.
• Built scripts and utilities for automating and streamlining many processes and written optimized HQL s and Pig jobs.
• Expert in analyzing complex problems and identifying root cause and providing solutions

Confidential

Hadoop Consultant

Environment: & Responsibilities: Cloudera CDH 5.0, YARN, Java, MapReduce, Talend, Hive, HBase, Pig, Sqoop, oozie, flume

Responsibilities:

• Architecting and designing end - to-end solution to store, process and analyze source data that comes from different types of data feeds.
• Designing and developing Map Reduce and Cascading programs to process the different type of inputs coming from
• RDBMS, Flat files (EBCDIC / ASCII) replacing the existing ETL code written in Ab Initio.
• Transformed multiple projects that were implemented in AbInitio into Talend Big Data Enterprise Version.
• Consistently got recognized by the management and customers for exceeding expectations
• Defined standards, best practices and design patterns to make the applications efficient, scalable and easily maintainable
• Built scripts and utilities for automating and streamlining many processes
• Expert in analyzing complex problems and identifying root cause and providing solutions

Confidential

Project lead.

Responsibilities:

• Involved in requirements gathering, source system analysis, writing extract requirements.
• Prepared detailed design specs for ETL development using Abinitio to support the development.
• Developed the graphs that interact with Comverse as well as SAP using Ab>Initio components like web service call, SAP RFC.
• Prepared unit test cases for all the graphs and built the prototype to test the whole process.
• Involved in setting up Hadoop infrastructure for one of the proof of concept in Horton work Sandbox and HDP.
• Analyzed the data from various resources.
• Involved import, exporting the data by using Sqoop.
• Involved in loading data into HDFS and written scripts to process the data using Pig Latin and also loaded the data to Hive managed tables and processed the data using Hive.

Environment: HDFS, MapReduce, Pig, Hive, Hortonworks HDP, Sqoop, Ab Initio 3.0, UNIX Sun Solaris, Comverse, Oracle, SQL

Confidential

Ab Initio Developer

Responsibilities:

• Involved in requirements gathering, source system analysis, writing extract requirements.
• Involved in designing the fact and dimension tables.
• Prepared detailed design specs for ETL development using Abinitio to support the development.
• Developed the load ready graphs and load graphs with required PCF scripts.
• Prepared unit test cases for all the graphs and tested the whole process.

Environment: Ab Initio (GDE1.14, Co>Op 2.14), UNIX Sun Solaris, Teradata, SQL, Maestro

Ab Initio Developer

Confidential

Responsibilities:

• Total New Work, Total Reviews, Total Closes, NRI Closes, Fraud App Close, ATO Close, Lost/Stolen Close, Inventory.
• Involved in requirements gathering, source system analysis, writing extract requirements.
• Preparing the design document, Coding and Unit Testing.

Environment: Ab Initio (GDE1.14, Co>Op 2.14), UNIX Sun Solaris, Teradata, SQL, MaestroClient:

Confidential

Ab Initio Developer

Responsibilities:

• Involved in requirements gathering, source system analysis, writing extract requirements.
• Prepared detailed design specs for ETL development using Abinitio to support the development.
• Developed the load ready graph and load graph with required PCF and PDF scripts.
• Prepared unit test cases for all the graphs and tested the whole process.

Environment: Ab Initio (GDE1.14, Co>Op 2.14), UNIX Sun Solaris, Teradata, SQL, Maestro

Confidential

Ab Initio Developer

Responsibilities:

• Involved in requirements gathering, source system analysis, writing extract requirements.
• Prepared detailed design specs for ETL development using Abinitio to support the development.
• Developed the load ready graph and load graph with required PCF and PDF scripts.
• Prepared unit test cases for all the graphs and tested the whole process.

Environment: Ab Initio (GDE1.14, Co>Op 2.14), UNIX Sun Solaris, Teradata, SQL, Maestro, Windows Vista

Confidential

Ab Initio Developer

Responsibilities:

• Developed the graphs for some interfaces with required PCF and PDF scripts.
• Prepared unit test cases for all the graphs and tested the whole process.
• Some graphs involves XML data.So XML components like read XML, writeXML components were used.

Environment: Ab Initio (GDE1.14, Co>Op 2.14), UNIX Sun Solaris, Oracle, SQL, Autosys, Windows Vista

Confidential

Ab Initio Tester / Developer

Responsibilities:

• Involved in requirements gathering, source system analysis, writing extract requirements, assisting in data model design for ALS (Advanced Loan System).
• Wrote detailed specs for ETL testing using Abinitio to support the passing of new accounts, and updates to existing collection accounts, from ALS to DM (Debt manager).
• Tested graphs to do complex calculation by using normalize and de-normalize components to load the data in RH and RA Updates.
• Developed Ab Initio graphs involving the changes to the Phase1 graphs and newly implemented the sub graph for all the validations part.
• Developed JCL to run the graphs in INT mainframe environment.
• Created Summary tables using Rollup and component.
• Prepared test cases for all the graphs and produced the snapshot results for further verification of the results.

Environment: Ab Initio (GDE 1.14, Co>op 2.14), Putty, DB2, IBM Mainframes, Microsoft Visio.