SUMMARY

• Has more than 12 years of experience in implementing and supporting data warehouses of multiple terabytes to peta bytes using tools like Microsoft Azure, Snowflake, Apache Spark, Informatica and languages like SQL, Python.
• Has more than 4 years of experience in developing data processing applications using Python
• Expertise in Microsoft Azure cloud to perform ETL & ELT operations on data and optimize performance by building robust, extensible and reusable framework.
• Experience in DevelopingSparkapplications usingSpark - SQL, PySparkin Data bricks for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming teh data to uncover insights into teh customer usage patterns.
• Strong knowledge in developing, debugging, fine tuning data processing jobs using ETL tools like Informatica and DMExpress
• Fine-tuned several complex ETL Reporting applications with a goal of providing faster and more efficient BI platform for business users.
• In-depth experience in translating key strategic objectives into actionable and governable roadmaps and designs using best practices and guidelines. Worked on all facets of software development life cycle.
• Successfully led executed several Simplification Process Optimization initiatives to bring efficiency into business processes.
• Experienced in performance tuning of Spark Applications for setting right Batch Interval time, correct level of Parallelism and memory tuning.
• Experience in enforcing Data Quality, Data Validation standards during teh data load process
• An TEMPeffective communicator with excellent relationship building interpersonal skills. Follow TEMPeffective time management and goal driven approaches. Strong analytical, problem solving organizational abilities
• Has strong relational Database concepts and worked on several databases such as SQL and DB2.
• Hands-On experience in designing and developing scalable applications to process data in multiple terabytes to petabytes using tools like Microsoft Azure, Azure Data Factory, Azure Databricks, Apache Spark, Apache Hadoop, Hive, Pig
• Experience in processing data from large files and preparing dashboards using python libraries like Pandas, Numpy, Matplotlib
• Microsoft certified Data Engineer Associate, Informatica and Oracle SQL certified developer

TECHNICAL SKILLS

Programming Languages: SQL, PL/SQL, Python, UNIX Shell Scripting

DBMS/Databases: Oracle 8i/9i/10g, Microsoft SQL Server, DB2, My SQL 4.x/5.x,Snowflake

Big Data Ecosystem: Apache Spark, Spark SQL, Hive, Pig, Hadoop

Methodologies: Agile, Water Fall.

ETL Tools: Informatica, DMExpress

Cloud Technologies: Microsoft Azure

PROFESSIONAL EXPERIENCE

Confidential

Technologies: Microsoft Azure, Azure Data Factory, Azure SQL database, Azure Databricks, Power BI

Responsibilities:

• Analysing different database options available and identifying teh right fit for teh needs
• Designing and developing ETL jobs to ingest inbound files using Azure Data Factory
• Developing scalable code in python to ingest multiple files and prepare them for loading into database
• Designing teh database for best performance
• Maintaining teh data pipelines in production, to make sure teh end users always has accurate data
• Reaching out to source teams to share requirements of dashboard and establishing pipeline to receive teh data on monthly basis

Confidential

Technologies: Azure Data Factory, Azure Databricks, Azure SQL databases, Snowflake, Python, SQL, Informatica

Responsibilities:

• Designing cloud migration strategies, data delivery architecture
• Determining appropriate tools to use for migration purpose
• Performs long-term evaluations of systems, databases, and solutions for security risks.
• Considers innovative methods and designs to continuously improve our technical capabilities / solutions
• Model processes to clarify technical designs, and to enhance or re-engineer business processes, prior to, or in parallel with, solution design and implementation, as necessary
• Building python data pipelines in cloud to meet business requirements
• Developing python scripts to copy data between on prem and cloud databases
• Built data pipelines using Spark framework that are extensible and reusable in cloud to ingest and transform structured/semi-structured data.
• Post migration data testing using tools like python

Confidential

Technologies: Informatica, SQL, SQL Server, Python, Apache Spark

Responsibilities:

• Designing and developing ETL jobs to ingest inbound files
• Supporting and improving teh existing data pipelines in production
• Developing scalable code in python to ingest multiple files and prepare them for loading into database
• Maintaining teh data pipelines in production, to make sure teh end users always has accurate data
• Performs long-term evaluations of systems, databases, and solutions for security risks. Considers innovative methods and designs to continuously improve our technical capabilities / solutions
• Model processes to clarify technical designs, and to enhance or re-engineer business processes, prior to, or in parallel with, solution design and implementation, as necessary
• Working on pilot programmes with vendors, to share teh data as per teh requirements
• Developing pipelines to share data with external vendors as per business needs

Confidential

Technologies: Informatica, SQL, SQL Server, Python

Responsibilities:

• Designing teh back end database system for teh application front end
• Designing and developing ETL jobs to support teh data load process
• Using Informatica PowerCenter Designer analyzed teh source data to Extract & Transform from various source systems (SQL server and flat files) by incorporating business rules using different objects and functions that teh tool supports.
• Using Informatica PowerCenter created mappings and mapplets to transform teh data according to teh business rules.
• Used various transformations like Source Qualifier, Joiner, Lookup, sql, router, filter, expression and update strategy.
• Implemented slowly changing dimensions (SCD) for some of teh Tables as per user requirement.
• Developed Stored Procedures and used them in Stored Procedure transformation for data processing and has used data migration tools
• Performance tuning teh jobs for better performance
• Maintaining teh data pipelines in production, to make sure teh end users always has accurate data within teh timeline as per teh SLA

Confidential

Technologies: Informatica, SQL, DB2, Hadoop, Hive, HortonWorks

Responsibilities:

• Understanding existing informatica jobs that loads data into DB2
• Acquiring knowledge about teh existing application in production and handling teh production issues and closing them within SLA
• Working on Proof of Concept, with Hadoop technology to improve teh data processing speeds
• Actively involved in gathering requirements for teh enhancements and upcoming projects
• Developing scripts to create and load data into Hive tables as per teh business rules
• Acquiring knowledge about teh existing application in production and handling teh production issues and closing them within SLA
• Providing impact analysis of teh requirements on current production system
• Analyzing teh data from different sources based on End user requirements and extensively involved in discussions with end users in decision-making.
• Represented/lead discussions related to product/application/modules/team (for example, leads technical design reviews). Builds relationships with internal customers/stakeholders. Represent teh team in front of teh customer
• Enforcing quality processes such as extensive testing, root cause analysis, code review to deliver teh quality product on time as per deadline
• Providing estimates, resource needs, milestones, risks as per teh requirements
• Developed ETL routines using Informatica Power Center and created mappings involving transformations like Lookup, Aggregator, Ranking, Expressions, Mapplets, SQL overrides usage in Lookups and source filter usage in Source qualifiers and data flow management into multiple targets using Routers.
• Extensively used Mapping Variables, Mapping Parameters to execute complex business logic
• Design and development of complex ETL mappings making use of Connected/Unconnected Lookups, Normalizer.
• Monitoring teh jobs on daily basis and undressing teh failures immediately.
• Involving in discussions with reporting team, to identify teh data fields and loading teh data accordingly

Confidential

Technologies: Informatica, SQL, DB2, Hadoop, Hive, HortonWorks

Responsibilities:

• Analysing teh data from different sources based on End user requirements and extensively involved in discussions with end users in decision-making.
• Represented/lead discussions related to product/application/modules/team (for example, leads technical design reviews). Builds relationships with internal customers/stakeholders. Represent teh team in front of teh customer
• Enforcing quality processes such as extensive testing, root cause analysis, code review to deliver teh quality product on time as per deadline
• Providing estimates, resource needs, milestones, risks as per teh requirements
• Developed ETL routines using Informatica Power Center and created mappings involving transformations like Lookup, Aggregator, Ranking, Expressions, Mapplets, SQL overrides usage in Lookups and source filter usage in Source qualifiers and data flow management into multiple targets using Routers.
• Extensively used Mapping Variables, Mapping Parameters to execute complex business logic
• Design and development of complex ETL mappings making use of Connected/Unconnected Lookups, Normalizer.
• Monitoring teh jobs on daily basis and undressing teh failures immediately.
• Involving in discussions with reporting team, to identify teh data fields and loading teh data accordingly
• Actively involved in gathering requirements and acquiring application knowledge from teh Business.
• Analyzed sources and End user requirements and extensively involved in discussions with end users in decision-making.
• Created ETL detail design document and ETL standards document.
• Developed ETL routines using Informatica Power Center and created mappings involving transformations like Lookup, Aggregator, Ranking, Expressions, Mapplets, SQL overrides usage in Lookups and source filter usage in Source qualifiers and data flow management into multiple targets using Routers.
• Extensively used Mapping Variables, Mapping Parameters to execute complex business logic
• Design and development of complex ETL mappings making use of Connected/Unconnected Lookups, Normalizer.
• Proficient in using Source Analyzer, Warehouse Designer, Transformation Designer, Mapping Designer and mapplet Designer.
• Used debugger in debugging some critical mapping by setting breakpoints and trouble shot teh issues by checking sessions and workflow logs.
• Involved in identifying bottlenecks in source, target, mappings and sessions and resolved teh bottlenecks by doing Performance tuning techniques like increasing block size, data cache size, sequence buffer length.
• Developed UNIX shell scripts to create parameter files, rename files, compress files and for prebalancing teh flat file extracts.
• Performed active interaction with teh client and effort estimation for teh new requirements in teh project.
• Responsible for teh documentations of teh different processes carried out like design documents and mapping documents, share point for teh version control of teh documents.
• Worked closely in setting up teh environment for various file transfer activities between teh systems using SFTP as teh file transfer protocol.

Confidential

Technologies: Informatica, SQL, Sybase, Oracle

Responsibilities:

• Responsible for teh design, development, coding, testing, debugging and documentation of applications to satisfy teh requirements of one or more user areas.
• Using Informatica PowerCenter Designer, 9.0.1, 8.6, DMExpress 7.1 (Task editor, Job editor, Server) analyzed teh source data to Extract & Transform from various source systems (oracle 10g and flat files) by incorporating business rules using different objects and functions that teh tool supports.
• Using Informatica PowerCenter created mappings and mapplets to transform teh data according to teh business rules.
• Using DMExpress created tasks and jobs to transform teh data from flat files according to teh business rules.
• Used various transformations like Source Qualifier, Joiner, Lookup, Sql, Router, Filter, Expression and Update Strategy in Informatica
• Used various tasks like Join, Sort and Merge in DMExpress.
• Implemented slowly changing dimensions (SCD) for some of teh Tables as per user requirement.
• Developed Stored Procedures and used them in Stored Procedure transformation for data processing and has used data migration tools
• Documented Informatica mappings, DMExpress tasks in Excel spread sheet.
• Tuned teh Informatica mappings for optimal load performance.
• Created and Configured Workflows and Sessions to transport teh data to target warehouse Oracle tables using Informatica Workflow Manager.
• This role carries primary responsibility for problem determination and resolution for each application system
• Worked along with UNIX team for writing UNIX shell scripts to customize teh server scheduling jobs.
• Constantly interacted with business users to discuss requirements.

Confidential

Technologies: Informatica, SQL, Sybase, Oracle

Responsibilities:

• Gatheird business requirements from Business Analyst.
• Designed and implemented appropriate ETL mappings to extract and transform data from various sources to meet requirements.
• Designed and developed Informatica ETL mappings to extract master and transactional data from heterogeneous data feeds and load
• Installed and Configured teh Informatica Client tools.
• Worked on loading of data from several flat files to XML Targets.
• Designed teh procedures for getting teh data from all systems to Data Warehousing system.
• Created teh environment for Staging area, loading teh Staging area with data from multiple sources.
• Analyzed business process workflows and assisted in teh development of ETL procedures for moving data from source to target systems.
• Used workflow manager for session management, database connection management and scheduling of jobs.
• Created UNIX shell scripts for Informatica ETL tool to automate sessions.
• Monitored sessions using teh workflow monitor, which were scheduled, running, completed or failed. Debugged mappings for failed sessions.