Data Engineer Resume
Huston, TX
SUMMARY
- Having around 8+ years of IT experience in Data Warehousing ETL - Development, Migration with major clients (Banking, Healthcare, Marketing) using Informatica-PowerCenter, IICS, SQL, Microsoft Azure Databricks & Unix Shell Scripting.
- Expertise in developing Informatica Cloud IICS Mappings, Mapping Tasks, Taskflows & generating API’s and load data into Cloud Database systems using Microsoft Azure Sql Server & Azure Databricks.
- Expertise on developing Cloud Data Quality transformations like Cleanse, De-duplicate, Labeler, Address verifier, standardization and invoke these transformations into CDI mappings as CDQ objects
- Strong working experience on Data Warehousing applications, directly responsible for the Extraction, Transformation and Loading of data from multiple sources into Data Warehouse. Extensively used Informatica client tools - Source Analyzer, Warehouse design.
- Expertise on all basic IICS transformations, tasks including Hierarchy Builder, Hierarchy Parser, data masking, Replication task, Rest V2 Connector & Web services transformation.
- Expertise in working with relational databases such as Oracle 11g/10g/9i/8x, SQL Server 2008/2005, and Teradata.
- Strong Experience in performance tuning of all ETL objects like optimizing ETL queries & optimizing ETL mappings using various techniques.
- Strong Experience in writing Unix shell scripts for triggering ETL workflows, generating parameter files, various file handling scenarios & file data corrections.
- Experience in resolving on-going maintenance issues and bug fixes; monitoring Informatica sessions as well as performance tuning of mappings and sessions.
- Experience in all phases of Data warehouse development from requirements gathering for the data warehouse to develop the code, Testing and Documenting.
- Extensively using Informatica PowerCenter tools such as Source Analyzer, Transformation Developer, Mapplet Designer, Mapping Designer, Workflow Manager, Workflow Monitor and Repository Manager.
- Working on transformations like Source Qualifier, Expression, Aggregator, Router, Filter, Update Strategy, Lookup, Sorter, Normalize, Union, Stored Procedure, and Sequence Generator, Slowly Changing Dimensions.
- Scheduling and Creating Workflows, sessions involved in the Performance Tuning of Mappings Sessions.
PROFESSIONAL EXPERIENCE
Confidential, Huston, TX
Data Engineer
Responsibilities:
- Followed Agile Practices in complete end-to-end project deliverables with 2-week Sprint schedule.
- Worked on all IICS transformations, tasks including Hierarchy Builder, Hierarchy Parser, data masking, Replication task, Rest V2 Connector & Web services transformation.
- Worked on different stages of SDLC life cycle (Planning, Requirements, Design, Build, Document, Test, Deploy, Maintenance).
- Data was extracted from source files and load them into MS-Azure cloud database. Optimized the Informatica objects & SQL queries for better performance of the ETL loads.
- Used mapping and workflow parameters and variables for extracting latest data based on last update date and last run id.
- Built up XML’s for extracting data from Landing to staging.
- Created new Database tables and designed appropriate key constraints & indexes and applied the business logic in SQ query as per the BRD Rules document.
- Developed new IICS taskflows, mappings with run-time target generation in cloud database and also migrated the power-center workflows to IICS platform.
- Performed IICS taskflow data loads to Azure Databricks & MS sql server cloud database systems.
- Developed Cloud Data Quality transformations like Cleanse, De-duplicate, Labeler, Address verifier, standardization and invoke these transformations into CDI mappings as CDQ objects.
- Created Unix shell scripts for triggering ETL workflows, generating parameter files, various file handling scenarios & file data corrections.
- Implemented SCD Type 1 and Type 2 methodologies in ODS tables loading to keep historical data in data warehouse.
- Expertise on using postman application to perform REST APIs call and execute IICS tasks using API.
- Automated the validations of the ETL objects using Data validation queries & Reconciliation process (Source vs Target).
- Designed incremental load process to load data into staging tables and load into final ODS Layer using incremental batch processing.
- Created Worklets, Workflow and Tasks to schedule the loads at required frequency using Workflow Manager and through Autosys Box setup using jil commands.
- Worked on initial phase of migrating/converting the Alteryx, informatica power Centre code components to IICS platform.
- Have Undergone Certifications in IICS cloud platform & Azure AZ-900 cloud computing.
Environment: IICS, CDQ, Informatica PowerCenter 10.4.1, Azure SQL Server Cloud DB, Shell Scripts, ORACLE 11g, UNIX, Alteryx, Autosys, UNIX
Confidential, Hartford, CT
Informatica Developer
Responsibilities:
- Responsible for Business Analysis and Requirements Collection.
- Worked on Informatica Power Center tools- Designer, Repository Manager, Workflow Manager, and Workflow Monitor.
- Parsed high-level design specification to simple ETL coding and mapping standards.
- Involved in building the ETL architecture and Source to Target mapping to load data into the Data warehouse.
- Created mapping documents to outline data flow from sources to targets.
- Extracted the data from the flat files and other RDBMS databases into the staging area and populated onto the Data warehouse.
- Maintained stored definitions, transformation rules and targets definitions using Informatica repository Manager.
- Used various transformations like Filter, Expression, Sequence Generator, Update Strategy, Joiner, Stored Procedure, and Union to develop robust mappings in the Informatica Designer.
- Developed mapping parameters and variables to support SQL override.
- Created mapplets to use them in different mappings.
- Developed mappings to load into staging tables and tan to Dimensions and Facts.
- Worked on different tasks in Workflows like sessions, events raise, event wait, decision, e-mail, command, worklets, Assignment, Timer, and scheduling of the workflow.
- Created sessions, configured workflows to extract data from various sources, transformed data, and loaded into data warehouse.
- Used Type 1 SCD and Type 2 SCD mappings to update slowly Changing Dimension Tables.
- Extensively used SQL to load data from flat files to the database tables in Oracle.
- Modified existing mappings for enhancements of new business requirements.
- Used Debugger to test the mappings and fixed the bugs.
- Wrote UNIX shell Scripts & PMCMD commands for FTP of files from remote server and backup of repository and folder.
- Involved in Performance tuning at source, target, mappings, sessions, and system levels.
- Prepared migration document to move the mappings from development to testing and tan to production repositories.
Environment: Informatica Power Center 10.2, PL/SQL, Oracle 10g/9i, Autosys, UNIX
Confidential, Stanford, CT
ETL Consultant
Responsibilities:
- Using Informatica PowerCenter Designer analyzed the source data to Extract & Transform from various source systems (oracle 10g, DB2, SQL server and flat files) by incorporating business rules using different objects and functions dat the tool supports.
- Using Informatica PowerCenter created mappings and mapplets to transform the data according to the business rules.
- Used various transformations like Source Qualifier, Joiner, Lookup, Sql Qualifier, router, Filter, Expression and Update Strategy.
- Implemented slowly changing dimensions (SCD) for some of the Tables as per user requirement.
- Developed Stored Procedures and used them in Stored Procedure transformation for data processing and have used data migration tools
- Documented Informatica mappings in Excel spreadsheet.
- Tuned the Informatica mappings for optimal load performance.
- Have used FLOAD, MLOAD Teradata utilities to export and load data to/from Flat files.
- Used Debugger to test the mappings and fixed the bugs
Environment: Informatica PowerCenter Designer 10.1, Oracle 10g/9i, DB2 6.1, Teradata, Red Hat Linux
Confidential, Dallas, TX
Informatica Developer
Responsibilities:
- Understanding the Business requirements based on Functional specification to design the ETL methodology in technical specifications.
- Consolidation, standardization for the unstructured flat file data.
- Responsible for developing, support and maintenance for the ETL (Extract, Transform and Load) processes using Informatica Power Center 9.6
- Wrote SQL-Overrides and used filter conditions in source qualifier thereby improving the performance of the mapping.
- Designed and developed mappings using Source Qualifier, Expression, Lookup, Router, Aggregator, Filter, Sequence Generator, Stored Procedure, Update Strategy, joiner, and Rank transformations.
- Managed the Metadata associated with the ETL processes used to populate the Data Warehouse.
- Implemented complex business rules in Informatica Power Center by creating reusable transformations, and robust Mapplets.
- Implemented performance tuning of Sources, Targets, Mappings and Sessions by identifying bottlenecks and used Debugger to debug the complex mappings and fix them.
- Improved session Performance by enabling property incremental aggregation to load incremental data into the target table.
- Worked with the Functional team to make sure required data has been extracted and loaded and performed the Unit Testing and fixed the errors to meet the requirements.
- Copied/Exported/Imported the mappings/sessions/ worklets /workflows from development to Test Repository and promoted to Production.
- Used Session parameters, Mapping variable/parameters and created Parameter files for imparting flexible runs of workflows based on changing variable values.
- Worked with Static, Dynamic and Persistent Cache in lookup transformation for better throughput of Sessions.
- Used PMCMD command to automate the Power Center sessions and workflows through UNIX.
Environment: Informatica Power Center 9.6, Oracle 10g, SQL Server 2005, DB2, SQL*Plus, SQL Loader, SQL Developer, Autosys, Flat files, UNIX, Windows 2000
Confidential, Irving, TX
ETL Developer
Responsibilities:
- Designed and developed Process Code update mapping by using Informatica Power Center 9.6
- Designed and developed Domain Check mappings
- Created One shot file document as per the new feed
- Created test plan documents and test scripts for one shot files and tested accordingly
- Regularly coordinating with Onsite and Offshore team
- Created mapping documents to outline data flow from sources to targets.
- Extracted the data from the flat files and other RDBMS databases into the staging area and populated onto the Data warehouse.
- Used various transformations like Filter, Expression, Sequence Generator, Update Strategy, Joiner, Stored Procedure, and Union to develop robust mappings in the Informatica Designer.
- Extensively used SQL* loader to load data from flat files to the database tables in Oracle.
- Used Debugger to test the mappings and fixed the bugs.
Environment: Informatica Power Center 9.6, SQL*Loader, Oracle9i/10g, AUTOSYS, Windows XP, UNIX