Data Engineer Resume
Tampa, FL
PROFESSIONAL SUMMARY:
- 10+ years of IT experience with strong background in Design, Analysis, Development, Maintenance, Enhancement, Implementation of Data Warehousing and Data Migration projects using ETL Tools like Informatica Technologies.
- Worked on key technologies and platforms including Informatica power center, IICS (Informatica Intelligent Cloud Services)/ICS (Informatica Cloud Integrations), MS SQL, SSIS, SSRS, Snowflake, Azure, Alteryx, MDM and IDQ .
- Solid experience in Informatica Master Data Management (MDM), Informatica Data Quality (IDQ) environment.
- Proficiently experienced on Mapping, Mapping Configuration, Data Synchronization and Replication Tasks and Task flows, mappings in I ICS which includes calling Scripts, adding parameter files.
- Experience in working with various data sources like Oracle, SQL Server, Azure platforms, DB2 and Worked on integrating data into/from flat files, CSV, MS Excel, COBOL, EDI and XML files.
- Extensively worked on developing Informatica mappings using different transformations, sessions and workflows . Debugging & optimizing the mappings using Power Center Designer and used innovative logics to provide maximum efficiency and expertise in Performance Tuning.
- Experience in direct and indirect file loading and experience in using mapping/session parameters & variables and parameter files and developing reusable mapplets, session, email, decision and controls tasks and support the incremental loads using parameter files & Pre, post session commands etc.
- Strong clutch on Dimensional Modeling, Entity Relation (ER) Diagrams, Normalization and De normalization, Star & Snowflake Schema, Fact & Dimensional tables, slowly changing dimension (SCD) and DW concepts. Familiar with Data profiling, Data validation (DV) and Change Data Capture (CDC) concepts.
- Experience in writing, testing and implementation of the SQL, T - SQL and PL/SQL code like triggers, stored procedures, functions, packages and very good at table partitioning and materialized views.
- Skilled at creating Stored Procedures to generate complex JSON data and Design JSON format based on given UI requirements.
- Very good hands on Source Control methodologies for ETLs and Database objects using Visual Studio (TFS), SVN Tortoise and GIT version control systems.
- Well accomplished with Agile tools like Microsoft VSTS for backlogs & MVP managements and very good at managing and work on Azure DevOps.
- Designing and Generating Analytical, Custom and Automated reports using Microsoft Power BI Desktop and publishing to business partners & leaders.
- Expertise in creating pipelines for sample reports and data validations using Alteryx based on adhoc requirements or enhancements.
- Good hands on Snowflake Cloud Data platform and snow SQL and strong knowledge on Cloud Data warehousing concepts.
- Experienced in Informatica admin activities like creating domain, node and those properties and start/stop Informatica services, creating and managing users and groups using admin console or using UNIX environment.
- Good skilled in UNIX shell scripting, job scheduling and error handling, FTP and file management in UNIX environments.
- Experienced with industry standard methodologies like Waterfall and Agile methodologies in the Software Development Life Cycle (SDLC).
- Good experience in Azure relational cloud database services and virtual machines.
- Excellent knowledge on other Informatica tools like power exchange, B2B data integration etc.
- Experienced in different types of test cycles like Unit testing, Integration testing, Regression testing, User Acceptance Testing (UAT) and implementing the respective modifications and fixing the issues.
- Proficient in interaction with the business users by conducting meetings, Conversations with the clients in Requirements Analysis phase and attending Sprint Planning, Code Reviews etc.
- Presented options for CI/CD Continuous Integration/Continuous Deployment using Different ETL Tools.
- Good self-learning capabilities and quick learner in new technologies and proved productive within short period of time. Team player, Motivated, able to grasp things quickly with analytical and problem-solving skills.
- Excellent analytical/ communication skills and good team player.
TECHNICAL SKILLS:
Databases: Oracle 11g/10g/12c, SQL Server 2008/14/17 (On premises & Cloud)
OLAP Tools: Business objects XI, Alteryx, Power BI, SSRS (SQL Server Reporting Services)
Languages: SQL, T- SQL, PL/SQL, Perl, Unix Shell Script
Data Modeling: Erwin 4.1, Star & Snowflake Schema Modeling, Fact & Dimensions
Other Tools: Toad (Oracle & SQL server), SSMS (SQL server Management Studio), SQL* Plus, Azure DevOps, Benthic, WinSCP, Putty, FileZilla, B2B Data transformation, Snowflake Cloud, Snow Sql
Operating Systems: Windows XP/7/8, Unix
Job Monitoring Tools & Schedulers: TWS (Tivoli workflow scheduler), Control-M, Power center Monitor
Agile Tools: VSTS, Asana, JIRA
PROFESSIONAL EXPERIENCE:
Confidential, Tampa, FL
Data Engineer
Responsibilities:
- Review business requirements and designing Integrations and database objects per specifications and standards and evaluate the designs with architects.
- Developing various mappings and tasks in Informatica using different transformations such as the Dynamic Lookup, SQL, Rank, Aggregator Joiner, Update strategy etc.
- Involved in debugging ETL issues and complex defect fixes and implemented the custom error control logic in ETL pipeline to capture & load the bad records to error tables/files and use/share these details for data analysis.
- Develop & executing Pre and Post commands and Unix Scripts in Mapping Configuration tasks in Informatica .
- Schedule the batches in Tivoli Workload Scheduler (TWS) and Create and maintain batch process and respective dependencies.
- Performance tuning of mappings by following ETL best practices and applied several methodologies/ partitioning to get good performance by decreasing the run time of workflows.
- Implementing several validation rules using various ETL transformations to cross verify the loaded data accuracy.
- Verifying the developed code with sample/test data using unit testing and system integration testing (SIT) and regression testing respectively.
- Writing SQL, PL/SQL codes, stored procedures, functions, triggers, packages and participated in query optimization and problem-solving techniques. Creating indexes and partitioned the tables to improve performance.
- Creating Stored Procedures to generate JSON data and Design JSON format based on given UI requirements.
- With Alteryx tool, generating sample reports for adhoc requirements & perform data validations.
- Building queries to populate/retrieve data silo to/from Azure cloud databases.
- Extensively working with MS SQL Studio 2017 & Toad SQL Server to querying the SQL Server database and writing the procedures and scripts and troubleshooting techniques.
- Designing and Generating Custom and Automated reports for business using Microsoft Power BI Desktop and publishing them to Project Leads and Partners.
- Created UNIX shell scripts to run the ETL jobs and handling file transfers.
- Involved in the User Acceptance Test (UAT) with end users to provide required data and support any enhancement suggested by the user.
- Participating Sprint Planning, Data architectural and ETL design sessions, Code Reviews, Sprint Retrospectives etc and actively involving on project priority tasks and sharing suggestions for best plans to implement.
- Maintaining developed code in best source control systems like GIT, creating pull requests, merging code to master branch and pushing to repository.
- Creating Change Requests (CR) to deploy the developed code into QA Phase and provided solutions for failed jobs.
- Maintaining written documentation to describe program development, logic, coding, testing, changes and corrections and enhancements.
- Attending Daily stand ups for regular work tracking with team, discuss impediments if any and work towards to get fast & better resolution.
Environment:: Informatica Power Center 10.1.1, SQL Server 2008/17, Oracle 10/11g, UNIX, Windows 7, Microsoft Azure, Putty 0.64, Alteryx, Azure DevOps.
Confidential, Tampa, FL
ETL Developer II
Responsibilities:
- Review business requirements and designing Integrations per specifications and standards and evaluate the designs with architects.
- Participating Sprint Planning and actively involving on ETL/Database designing sessions.
- Designing and performing data analysis using sample/raw source data to implement valid requirements and Standardized the data to increase the data accuracy.
- Designing the ETL code at HLD (High Level Designing) and LLD (Low Level Designing) and involving business meetings.
- Developing various mappings and tasks in ICS using different transformations such as the expression, Lookup, Filter, Sorter, Router, Aggregator, SQL, Joiner, Update strategy and SQL transformation etc and worked on performance tuning with best practices.
- Implemented several Data Synchronization Tasks, Mapping Configuration Tasks, Task flows in ICS/IICS.
- Creating various connections using available connectors, importing power center mapplets, Scheduling tasks in ICS/IICS.
- Involved in debugging for existing ETL issues and using the flexible error handling logic within the pipeline to capture and load the bad records to error tables or error files and different job streams are maintained by control tables.
- Develop & executing Pre and Post commands and Unix Scripts in Mapping Configuration tasks in ICS.
- Extensively worked on power center tools for ETL design, development and running jobs, monitoring and troubleshooting etc.
- Tuning the mappings by following standards and best methods.
- Developing slowly changing dimensions (Type-2) to maintain historical data in EDW using effective date concept.
- Partitioning of ETLs and Database objects to improve performance.
- Worked extensively on Informatica mapping partitioning while running the jobs with huge volumes of data and partitioned the tables in database for optimal performance.
- Maintaining written documentation to describe program development, logic, coding, testing, changes and corrections and enhancements.
- Verifying the developed code with sample/test data using unit testing and system integration testing and regression testing respectively.
- Writing SQL, PL/SQL codes, stored procedures, functions, triggers, packages and participated in query optimization techniques and Problem-solving techniques.
- Good experience in Azure relational cloud database services and virtual machines.
- Working with MS SQL Studio 2017 to querying the SQL Server database and writing the procedures and scripts and troubleshooting techniques.
- Created UNIX shell scripts, PMCMD commands and retrieving & transferring files on remote servers and bridging between servers.
- Involved in the Defect analysis call for UAT environment along with users to understand the data and to make modifications if suggested by the user.
- Participating in code reviews, where actively involved to present developed code to review/verify informatica ETL standards in development and shared if any suggestions.
- Request for Change (RFC) to deploy the developed code into QA Phase and provided solutions for failed jobs.
- Attending Daily stand ups for regular work tracking with team, discuss impediments if any and work towards to get fast & better resolution.
Environment:: ICS (Informatica Cloud Services), IICS (Informatica Intelligence Cloud Services) Informatica Power Center 10.1.1, SQL Server 2008/17, Oracle 10/11g, UNIX, Windows 7, Microsoft Azure, Putty 0.64.
Confidential, Tampa, FL
ETL Developer II
Responsibilities:
- Working with business analyst and gathering required transformation rules and designing the ETL mapping documents, based on business requirements.
- Designing and performing data analysis using sample/raw source data to implement valid requirements and Standardized the data to increase the data accuracy.
- Actively contributing on ETL designing for better accomplishment and per standards and discussing with architects.
- Implementing several mappings using power center designer with transformations such as the Expression, Transaction Control, Lookup, Filter, Aggregator, SQL, Normalizer, Joiner, Update strategy and SQL transformation etc and perform performance tuning.
- Involved in troubleshooting for existing ETL bugs and using the custom error control logic with in the pipeline to capture and load the bad records to a control table.
- Used Informatica Power Center Workflow manager to create sessions, different Tasks like email, assignment, command tasks and creating batch jobs to run with the logic embedded in the mappings.
- Executed Pre and Post session commands on Source and Target systems and various command tasks using Shell Scripting. Support Incremental loading using parameter file.
- Altering mappings by following ETL best practices to improve the performance with several techniques to get good quality & performance by decreasing the run time of workflows.
- Adding various data cleansing rules using different ETL transformations to validate the loaded data accuracy.
- Coded dimensional modeled integrations like slowly changing dimensions (Type-1,2) to have transactional data in EDW.
- Partitioning of Informatica mappings to run the jobs with massive amount of data and partitioning DB objects for finest performance.
- Performing Record Count Verification DWH backend/Reporting queries against source and target as an initial check.
- Documenting program development, logic, coding, testing, changes and corrections and enhancements detailly.
- Validating the established code with unit testing and system integration testing and regression testing etc.
- Creating SQL, PL/SQL codes and DB objects like procedures, functions, triggers, packages and applying query optimizing methodologies.
- Extensively worked on Microsoft Management Studio 2008/14 to develop views, functions and SQL scripts and querying backend tables.
- Performed various User Acceptance test and system analysis call with business users.
- Working with Quality Assurance team to transition code details based on business requirements and defect remediations and creating Change requests.
Environment:: Informatica Power Center 9.6.1, SQL Server 2008/2014, Oracle 10/11g, UNIX, Windows 7, Putty 0.64, OBIEE 11.1.1.7.02
Confidential, Tampa, FL
ETL Senior Specialist
Responsibilities:
- Defining the specifications and designing the data models, based on business requirements.
- Designing and performing the Data Profiling, Data analysis using Data Quality Tools to implement Standardized the data and to increase the data accuracy.
- Designing the ETL code using Informatica, modifying the data using different transformations to achieve the client specific format in designer and load extracted data into Database.
- Developed various mappings using the transformations such as the expression, Lookup, Filter, Sorter, Router, Aggregator, Normalizer, Joiner, Update strategy etc and perform performance tuning.
- Extensively used Informatica debugger to figure out the problems in mappings. Also involved in troubleshooting existing ETL bugs and Designed the custom error control logic with in the pipeline to capture and load the bad records to a control table.
- Successfully configuring and managing the MDM HUB, which includes implementing hierarchies, relationships types, packages and profiles by using hierarchies’ tool in model workbench.
- Configuring the MDM Hub by designing & creating the Landing, Staging and Base objects and perform load process and generate valid data using different cleans functions.
- Creating and applying the Trust, Validation Rules and perform Match and Merge to generate the master data using exact match and fuzzy match logic types
- Extensively used Toad for SQL scripts and worked on SQL for enhancing the performance of the conversion mapping.
- Tuned the performance of mappings by following Informatica best practices and also applied several methods to get best performance by decreasing the run time of workflows.
- Written documentation to describe program development, logic, coding, testing, changes and corrections and enhancements.
- Validation of developed code with data using unit testing and integration testing and non-regression testing independently.
- Implementing the code for new countries of existing clients and maintenance of respective jobs.
- Scheduling the workflows by weekly/monthly loads using orchestration process developed in Perl and prepared Perl scripts to email the Job failure notice with error message and count of records.
- Writing SQL, PL/SQL codes, stored procedures, functions, triggers, packages and participated in query optimization techniques and Problem solving and troubleshooting techniques.
- Created deployment groups, migrated the code into different Informatica environments (Test, staging, production).
- Involved in the Defect analysis call for UAT environment along with users to understand the data and to make any modifications if suggested by the user.
- Supported the applications in QA Phase and provided solutions for failed jobs.
- Involved in production activities to make sure all issues are fixed in the respective turn-around times.
Environment:: Informatica Power Center 9.5, Informatica Data Quality 9.5, Informatica MDM 9.5, Oracle 10/11g, Perl, UNIX, Windows 8, Putty.
Confidential , Cambridge, MA
Senior ETL Developer
Responsibilities:
- Involved in requirement gathering and analysis of source data as data comes from different Source systems.
- Data Quality Analysis, Data Profiling & Requirement Analysis from Business. Analysis, Design, Development and testing of implementations.
- Participated in profiling and Data Quality of customer data to find out anomalies in the data using Informatica Data Quality (IDQ).
- Configuring the MDM Hub data model and creating the landing, staging data models, Queries and packages based on business requirements.
- Designing and developing the ETL code using Informatica to integrate the data from heterogeneous sources and implementing business requirements using different transformations in designer to load extracted customer data into landing tables.
- Check for missing data, negatives and consistency. Field-by-Field data verification has been done to check the consistency of source and target data.
- Prepared the error handling document to maintain the Error handling process (Error code and error table information).
- Extensively creating Cleanse functions and MDM Mappings. Configuring trust decay algorithms and Match & Merge set up to achieve the customer master data.
- Working extensively on SQL queries as part of the process to develop several requirements and handle different scenarios.
- Developing PL/SQL codes, stored procedures, functions, triggers, packages in Oracle.
- Creating and modifying several UNIX shell Scripts according to the changes & needs of the project or client requirements.
- Validation of developed code with data using unit testing and system integration test cycles.
Environment:: Informatica Data Quality 8.6, Informatica Power Center 8.6, Informatica/ Siperian MDM 9, Oracle 10/11g, UNIX, Windows XP.
Confidential, Philadelphia, PA
Senior Developer
Responsibilities:
- Developed ETL procedures to translate business rules and functionality requirements into business logic, ensuring lack of redundancy and compliance with standards.
- Used transformations such as the source qualifier, aggregators, connected & unconnected lookups, filters, sorter, stored procedures, router & sequence generator.
- Developed Reusable Mapplets and Transformations for standard business calculations.
- Extensively used various Performance tuning Techniques to improve the session performance.
- Improved workflow performance by shifting filters as close as possible to the source and selecting tables with fewer rows as the master during joins.
- Created events, decision and tasks in the workflows using workflow manager.
- Implemented performance tuning by using lookup caches, using tables with fewer rows as the master table in joiner transformations, dropped indexes and re-created them after loading data to targets and increased the commit interval.
- Executed sessions, sequential and concurrent batches for proper execution of mappings.
- Executed Pre and Post session commands on Source and Target database using Shell Scripting.
- Scheduled Sessions to update the target data using workflow Manager of Informatica and Involved in test case preparation.
Environment:: Informatica Power Center 8.6, Oracle 9i, SQL Server2005/2008, benthic SQL plus, Windows XP/NT, Putty.
Confidential, Dallas, TX
ETL Developer
Responsibilities:
- Analyze the specification provided by the client
- Performed Data Cleansing operations on the jobs using data cleansing functions.
- Data loading using INFORMARICA Power center Designer
- Worked on several transformations such as Filter, Rank, Sequence Generator and Expression Transformations in mapping Designer & creating the reusable transformations, designing the reusable mappings in Informatica.
- Performed Unit Testing for developed code.
- Handled the change request and the coding related issues in Oracle 9i in various modules.
- Involved in various critical analysis of sales data.
Environment:: Oracle 9i, Business objects XI, Informatica Power Center 7.1 client tools, Windows NT.