SUMMARY

• Over 11+ years of IT Exp in DW/BI. Involved in all SDLC phases.
• Vast experience in designing, building and implementing Cost effective, Scalable, Distributed, Fault Tolerant and high - performance data driven applications on Bigdata/Teradata/Azure.
• Hands on experience on cutting edge technologies like Azure, Bigdata, PySpark, Snowflake, Hive, Pig, Teradata, Talend, DataStage, RabbitMQ and Python.
• Certified in Teradata Development, Python3 Programming, Spark 2.4.
• Strong analytical capabilities, logical thinking and excellent communication skills

PROFESSIONAL EXPERIENCE

TECHNOLOGY LEAD

Confidential, San Francisco

Responsibilities:

• Design, build and implement real time data consumption and data processing framework using RabbitMQ, PySpark and Hadoop eco systems.
• Designed and developed process to consume real-time product related messages and load into event buffers.
• Designed, built, and implemented pySpark framework to read events-thin messages (API’s), extract documents (thick messages) and loads into Hive raw layer.
• Developed Spark framework to clean, filter, transform raw data and load into Hive base layer.
• Developed Talend framework to read newly adopted changes and propagate to downstream applications.
• Involved in creating and implementing logical and physical data modelling.
• Worked with Product owner, Scrum master, Data Architects and stake holders, implemented the product in Agile methodology.
• Experienced in working with the technologies like Python, Spark, Hive, HDFS, Talend, Teradata, and RabbitMQ.

LEAD BIGDATA Developer

Confidential, San Francisco

Responsibilities:

• Design and build metadata driven framework to sync customer data between HDFS and Azure Blob.
• Developed process to push data from HDFS to Azure using Distcp utility.
• Designed and created framework to ingest Azure Blob data to Snowflake warehouse.
• Implemented the restartbility logic to avoid rerunning the successfully completed process.
• Worked closely with Data science team to make sure data and its quality meeting the business requirements.
• Gained Hands on experience with Azure, Snowflake and Python scripting.

Senior BIG DATA Developer

Confidential, San Francisco

Responsibilities:

• This program aimed to pull Online, Retail and Franchise data from various sources into Hadoop Data Lake, integrate and provide single window view which enables business users to access the data of all channels.
• Partnered with Product owners, Technical Managers, Support teams, Data Architects to understand the use of their respective Data Marts by the business. actively participated in defining the scope of work, sizing the epics, writing user stories, preparing functional, technical documents.
• Designed, built and implemented metadata driven frame work to pull data from various source systems - MySQL, Oracle, Teradata, DB2 and XML Files into Hadoop using Talend, MySQL, HDFS, Hive and Pig.
• Created the framework in such a way to support the data ingestion in Overwrite/Append/Update modes.
• Wrote multiple Hive and Pig scripts to standardize, transform, aggregate the data and load into base layer.
• Worked with Hive managed, external and partitioned and non partitioned tables.
• Preparing support documents, involved in transitioning the application to Support teams. Mentored and trained the junior resources.

LEAD Developer

Confidential

Responsibilities:

• This project aimed to de-commission all legacy Teradata Tables, scripts, jobs in order to free up unused data and avoiding usage of processor for loading the unwanted tables.
• Analyzed all Bteq, Fastload, Multiload, and fastexport scripts to list out in which legacy tables being populated.
• Alerted all the downstream systems, Data Analysts about the disconnect of legacy data feed from Teradata.
• Provided the remediations to downstream applications, which still driven by legacy data.
• Closely worked with Data Architects, product owners in providing the remediations.
• Removed the loading jobs from CAWA, revoked access to all the users on legacy tables, removed the scripts, dropped the tables.
• Extensively used Teradata Queryband to identify the process which are using or feeding Legacy data.

LEAD Developer

Confidential

Responsibilities:

• This program aimed to flip the hierarchy of product from Corp/Comp (Corporation, Company) to BMC (Brand, Market, Channel) in EDW.
• Partnered with Data Architects, Technical Managers to analyze and understand the requirements. Prepared HLD and technical documents.
• Designed, build and developed DataStage integration platform to read BMC source XML files and loads into Teradata staging layer.
• Wrote fastload, mload and bteq scripts to push the data from BMC files into Teradata stage and base dimension tables.
• Changed all Fact table schemas to incorporate BMC details. Modified all fact load bteq’s to replace old hierarchy columns with new hierarchy columns.
• Replaced all reporting views to tag products with new hierarchy details.
• Extensively worked on DataStage, Teradata utilities like Fastload, Mload, Fastexport and Bteqs. Scheduled the batch process using CAWA.
• Lead the team of size:5. Attending daily Scrum calls, Co ordinating and delegated offshore work with Onsite managers, making sure deliverables in line and meeting client expectations in terms of code delivery, data quality.
• Extensively worked on DataStage, Teradata, and Shell Scripting.

SENIOR Developer

Confidential

Responsibilities:

• The objective is to Develop, Enhance, support customer account DataMart.
• Involved in all phases of SDLC from requirement gathering, designing, development, testing, and rollout to the end user and support for production environment.
• Executed the project in Agile methodology. Writing user stories, sizing, story prioritization. Attending Daily scrum call sharing the updates with Scrum master. Proactive intimation of road blocks and providing solutions to overcome.
• Writing Bteq/Fastload/Multiload and Tpt scrips to extract and transform the data to comply with business requirement, Unit Testing, validations and code migrations for the next level of testing. Acknowledging defects and fixing.
• Defining proper Primary Index (PI) taking into consideration of both planned access of data and even distribution of data across all the AMPS.
• SQL Tuning by choosing proper indexes, Collecting Stats, Creating Secondary/Join Indexes, Table partitioning and rewriting SQL’s
• Preparing cutover plan, involving in migration activities, provided warranty support.
• Informatica and Teradata widely used for the project implementation.