We provide IT Staff Augmentation Services!

Etl Architect / Lead Data Engineer Resume

0/5 (Submit Your Rating)

South, CarolinA

SUMMARY

  • 11+ years of strong IT experience with sound knowledge and experience in different phases of software development life cycle (SDLC) including analysis, design, development, testing and production. Strong understanding of relational database design concepts and in - depth knowledge of OLAP solutions.
  • Extensively worked on Informatica PowerCenter 9.x & 10.x, IICS, Oracle, SQL, PL/SQL, RedShift, Teradata, Athena, Postgres, Unix, Python.
  • Worked as a Team Lead with offshore team (offshore and onsite model).
  • Designed & implemented various data integration solutions and Best Practices for ETL mechanism using various ETL tools.
  • Good understanding of RDBMS concepts and experienced in writing queries using Oracle, Redshift, Teradata.
  • Extensively worked on integrating data from heterogeneous sources to a centralized data warehouse using Informatica PowerCenter and databases like Oracle, Redshift, Teradata, Athena, MS SQL Server and Flat Files (fixed width delimited).
  • Expertise knowledge in Performance Tuning. Extensively worked on optimizing the SQL’s (Hints, Indexes, Analytical functions, Temporary tables), Informatica (Partitions, Persistent Cache, Bulk Load, Push Down Optimization).
  • Extensively worked on preparation and maintenance of ETL documentation, including source-to-target mappings and business-driven transformation rules.
  • Experience in working with Informatica Data Quality (IDQ) for data and quality analysis.
  • Extensive experience with Oracle in working with Packages, Stored Procedures, Triggers, Cursors, Indexes and Functions.
  • Extensive experience with AWS with Redshift, S3 bucket, AWS Glue, Athena DB.
  • Experience in code migration from lower environments to Test and Test to Production. Prepared execution plan, run book, Database Scripts and Back out plan.
  • Solid development experience in Informatica Power Center to implement complex business scenarios and extensive knowledge in performance Tuning.
  • Ability to determine root cause of technical issues either on own or in a collaborative scenario. Strong problem-solving skills are necessary.
  • Good experience with Datawarehouse concepts, Star Schema, Snow Flake Schema.
  • Good experience with Unix Shell script. Developer wrapper script, archival script and file validation scripts
  • Experience in Scheduling Informatica sessions for automation of loads in Tivoli.
  • Expertise in Healthcare and Automotive domain.
  • Extensive experience in SDLC implementing Agile, SCRUM and Waterfall methodologies.
  • Well verge with Client Partner Relationship to grow footprints as a vendor.
  • Proficient in leading a team working from different locations and projects.
  • Involved in upgrading Informatica code from 10.2 to 10.5 version. Created model repository, validated the DB Connections, changes to odbc.ini file, updated the database drivers.
  • Good experience with Datawarehouse concepts, Star Schema, Snow Flake Schema
  • Experience with Water Fall and Agile Methodologies.
  • Experience working in the Functional Domains of Pharmaceuticals, Finance, Human Resource, Sales and Marketing.
  • Excellent technical and analytical skills with clear understanding of design goals of ER modeling for OLTP and dimension modeling for OLAP.
  • Good understanding of RDBMS concepts and experienced in writing queries etc using Oracle, MS SQL Server, Teradata and IBM DB2 UDB..
  • Ability to meet deadlines and handle multiple tasks, decisive with strong leadership qualities, flexible in work schedules and possess good communication skills.
  • Strong data analysis skillstaking pride in making data-driven decisions while defining and building the necessary tools to gather, track and measure key business data points.
  • Excellent communications skills able to present and discuss data models and analysis to a range of audiences, in a concise and effective manner
  • Ability to determine root cause of technical issues either on own or in a collaborative scenario. Strong problem-solving skills are necessary.

TECHNICAL SKILLS

RDBMS: ORACLE 8i/9i/10g/11g,12c, Redshift, Teradata, Athena, Postgress

O/S: Win XP/NT/ 98/95/2000/2003/2007 , 11, UNIX.

Languages: SQL, PL/SQL, Python, Shell Scripting

ETL Tools: Informatica 8.x,9.x, IICS, AWS Glue

PROFESSIONAL EXPERIENCE

Confidential, SOUTH CAROLINA

ETL Architect / Lead Data Engineer

Responsibilities:

  • Off shore Senior Tech lead and POC. Responsible for planning, executing the project, estimation, delivering the robust and optimum code.
  • Participated Team & BU meetings to gather functional and reporting requirements, document use cases, and analyze data and data sources to work on given tasks to fulfill the project requirements.
  • Interacted with client on gathering requirements, proposed the feasible solutions, designed, developed and reviewed the ETL’s to integrate data from various heterogeneous sources (Oracle, Postgres, Athena, S3 Buckets, Flat Files) to a Datawarehouse built on Redshift database using Informatica PowerCenter, IICS and AWS Glue.
  • Wrote New Epic, Story, Sub Task, and managing all status from Backlog to Close (Backlog, Ready, In progress, Ready for Acceptance and Close).
  • Worked with stake holders and ETL to formulate the transformation rules for ETL development that best suites the reporting needs.
  • Involved in requirement gathering from Business Users and Developed Technical Documentation for the Business and Team.
  • Analyzing the specifications provided by the clients and developing specification documents for reports as per client requirements.
  • Participates in regular status meetings, Release Management meetings and Design review sessions.
  • Designed and customized data models for Data warehouse supporting data from multiple sources on real time.
  • Created mapping documents to outline data flow from sources to targets.
  • Worked with Power Center designer tools (Source Analyzer, Target Designer and Mapping Designer), project team to design and implement the Informatica workflows.
  • According to business need, implemented SCD Type1, Type2 and Type3 to capture data history.
  • Involved on code deployment to QA, Pre-Prod and Prod environment.
  • Modifying existing shell scripts to full fill the new requirements.
  • Created/modified python scripts as per requirements.
  • Created and tested the Informatica reusable components that can be used across the projects and applications.
  • Unit Testing, User Acceptance Testing, Production Deployment and Post Production Support.
  • Created shell scripts to delete the 30 day's older logs historical files from S3 bucket passing year and year/months as parameter.
  • Analyzed the long running ETL jobs, identified the bottlenecks and fine-tuned the ETL’s.
  • Use of EXPLAIN PLAN, ANALYZE, HINTS, PARTITIONS to tune queries for better performance.
  • Developed reports in a consistent best practices format
  • Involved in Informatica upgrade from 10.2 to 10.5.
  • Migrated the code and data from environment to environment, created the ODBC connections, Application connections to load data to redshift on AWS environment.
  • Provided support for deployment activities (Created release note and Service now Ticket to move Informatica code to upper Environments).
  • Provided documentation as required (Created run book document).
  • Created check list, code review documents and uploaded into confluence page.
  • Lead a team size of 5 members and organized daily status and weekly status call meetings with off shore team.
  • Worked in Agile methodology.

Environment: Informatica PowerCenter 10.2, 10.5, IICS, AWS Glue, Redshift, Oracle, Athena, Postgres, Athena, AWS S3 bucket, Python, Unix Shell scripting, Tableau.

Confidential, California

Sr. ETL Developer

Responsibilities:

  • Analyzed various source systems/claim adjudication tools.
  • Involved in Meetings with Business analysts to understand the functionality, gather the requirements, and alleviate any issues and ambiguities in the requirements.
  • Lead a team size of 10 members and organized daily status and weekly status call meetings with off shore team.
  • Parsed high-level design specification to simple ETL coding and mapping standards.
  • Designed the ETL process by preparing the Low level, Physical solution design documents and other ETL Design specifications.
  • Involved in building the ETL architecture and Source to Target mapping to load data into Data warehouse.
  • Worked on Informatica Power Center tools- Designer, Repository Manager, Workflow Manager, and Workflow Monitor.
  • Worked with stake holders and ETL to formulate the transformation rules for ETL development that best suites the reporting needs.
  • Written scripts to dump data from flat files to oracle tables using SQL Loader.
  • Written Stored Procedure to implement SCD-1 and SCD-2 to load the data into stage tables from various system to a centralized repository by dynamically generate the Merge statements.
  • Created mappings, sessions to load claims data from various source systems built on multiple applications to Staging and then from Staging to Reporting in either TYPE-1 or TYPE-2 Approach.
  • Created Packages integrating multiple procedures.
  • Created functions to validate the data.
  • Developed Unix scripts to validate the files. Modified the Wrapper scripts and archival as per the requirements.
  • Implemented Control Mechanism that captures the Source, Stage and Report metrics and abort the ETL load process in case of metric match.
  • Involved in Unit Testing, created minus queries to validate data between tables in same databases and created views that compare Stage and Reporting.
  • Involved with project support teams like Informatica Admin team and DBA’s for project activities and getting tasks done within SLA’s.
  • Automated the ETL load process by scheduling the ETL jobs via Tivoli.
  • Involved in code migration process to higher environments and code validation after the code migration.
  • Reviewed the ETL’s developed by team to ensure the optimum, feasible code is implemented and best coding standards are practiced.
  • Written SQL queries to fix the historical data.
  • Prepared migration document to move the mappings from development to testing and then to production repositories.
  • Involved in performance tuning. Loaded the historical data in chunks, implemented Informatica partitions, restricted data at various transformations, used Hints.
  • Given transition to the production support team during the project handover phase.
  • Provided resolutions to the issues in Production and resolutions for defects.
  • Given KT to the production support team during the project handover phase.
  • Provided resolutions to the issues in Production and resolutions for defects.

Environment: Informatica PowerCenter 9.5, Oracle,Teradata, Python, Unix Shell scripting, Tableau.

Confidential, California

Informatica Developer

Responsibilities:

  • He interacted with business team during the requirement gathering phase.
  • Analyzed the Informatica session logs to identify the bottlenecks at the source, transformation and Targets.
  • Modified the Informatica mappings to change the load type from truncate and load to insert else update to ensure the number of records processed per day are limited.
  • Limited the Lookup transformations by moving the lookups to Source Qualifier with left outer join to eliminate the excess caches built.
  • Extensively worked on debugging Informatica mappings, mapplets workflows and sessions
  • Optimized the SQL’s at Source Qualifier, Lookup, Pre-SQL and Post SQL.
  • Created oracle procedures to create and drop the indexes/constraints.
  • Deeply involved in database design and Oracle PL/SQL objects (like Stored Procedures, Functions, packages) creation.
  • Created the approach document that describes the bottle necks in the existing code and the best approach that can be implemented to overcome the bottlenecks.
  • Involved in Performance tuning at source, target, mappings, sessions, and system levels.
  • Created the ETL Specification/ Design documents that have the detail description of Informatica and database objects used.
  • Involved in the impact analysis when the reusable Informatica objects were edited.
  • Reviewed the code done by other developers to ensure that all bottlenecks are eliminated and all the coding standards were followed.
  • Involved in unit testing and created Unit test case and Unit results.
  • Involved in Pre-Implementation and Post Implementation activities of production release.
  • Create and modified existing UNIX shells scripts to execute batch program.

Environment: Environment: Informatica PowerCenter 9.2, Oracle, Unix Shell scripting.

Confidential, Arizona

Informatica Developer

Responsibilities:

  • Involved in requirement gathering, Translates business requirements into viable technical solutions.
  • Analyzing the specifications provided by the clients and developing specification documents for reports as per client requirements.
  • Created mapping documents to outline data flow from sources to targets.
  • Developed and modified existing Packages, Stored Procedures, and Functions.
  • Analyzed current reports based on underlying SQL and documented them.
  • Created the design document, technical design document and user test cases.
  • Inserted data from different data sources to meet the needs of the client.
  • Written shell scripts to clean up files in temp, log and archival directory that are 30 days old and size greater than 1GB.
  • Created Workflows and scheduled for every 30 min to run the equipment integration workflow.
  • Import Source and Target Table from Database.
  • Generate and execute SQL for Target Tables.
  • Created and modified existing shells scripts.

Environment: Environment: Informatica PowerCenter 9.2, Oracle, Unix Shell scripting.

We'd love your feedback!