Etl Architect And Tech Lead Resume
New York City, NY
SUMMARY
- Highly Motivated and Result Oriented IT Professional with 14+ years of experience as an ETL Architect/Lead/Senior Developer in Software Development and Enhancement projects.
- Played various roles from Developer to Architect and Team Lead and have got extensive experience in ETL Informatica Power Center, Oracle SQL/PLSQL, Redshift and UNIX Shell Scripting
- Involve in the ETL Architectural design using layered approach which simplify the data load process
- Design the strategy for Error Handling, Restartability, Change Data Capture (CDC mechanism) and Data Lineage
- Design, Develop and Implement ABC (Audit Balance and Control) framework in ETL Jobs
- Define and Implement ETL Development Standards, Procedures and Best Practices
- Good exposure in all phases of ETL life cycle including Planning, Analysis, Design, Development, Implementation and Maintenance using Data warehouse and Data Marts
- Involve in the Data modeling of Data Warehouse and ER Relationship models with data modelers
- Develop solution in highly demanding environment and provide hands on guidance to other team members
- Hands on experience in handling/processing large volume of data (TBs) in business critical environments
- Good in Data Analysis, Data Profiling, Data Cleansing, Data Management, Troubleshooting and Performance tuning in various level of ETL Process
- Good hands on experience in IDQ using Informatica Developer tool, in ETL using Informatica PowerCenter tool, in CDC using Informatica PowerExchange tool and in MDM using Informatica MDM tool
- Involve in the Requirements Gathering with business teams positioned across multiple countries, Analysis, Design, Development and involvement in Unit testing, System testing, UAT & regression testing of SDLC life cycle
- Experience working in both Agile and Waterfall methodology
- Over 2+ experience in Qlikview reporting tool as a developer and designer
- Over 1 year experience in Business Object reporting tool as a developer
- Good knowledge of Data warehouse concepts and principles
- Star Schema, Snowflake, SCD, Surrogate keys, Normalization/De normalization.
- Good understanding and Implementation of Various Fact and Dimension Tables.
- Persuasive communicator with exceptional relationship management skills with the ability to relate to people at any level of business and management; highly ethical, trustworthy and discreet.
TECHNICAL SKILLS
ETL Tools: Informatica PowerCenter 9.6.1/9.1/8. x, Informatica Developer 9.1, Informatica PWX 9.1, Informatica MDM 9.1
Reporting Tools: Qlikview 11/9, Business Object 5.1
DB Tools: PLSQL Developer, TOAD, SQL Server Management Studio 2012/2010, Teradata SQL Assistant
Other Tools: MS Visio 2010, MS Power Point 2010
Data Modeling: ER Model, Star - Schema Model, Snowflake Model
Databases: Redshift 1.0, Oracle 11g/10g/9i, SQL Server 2012/10, Teradata 14
Operating Systems: UNIX, Windows
Languages: UNIX Shell Scripting, PL/SQL, SQL
SDLC Methods: Water Fall Methodology, Agile Methodology
PROFESSIONAL EXPERIENCE
Confidential, New York City, NY
ETL Architect and Tech Lead
Responsibilities:
- Planned for the migration of Informatica jobs from MicroStrategy Cloud with target database Netezaa to AWS (Amazon Web Services) cloud with target database Amazon Redshift
- Successfully completed migration of Informatica jobs from MicroStrategy Cloud to AWS cloud after analyzing and fixing several issues
- Involve in the high level ETL design changes and implement it in various phases
- Discuss with Business Analyst and Development team to review and finalize the requirement documents and source to target mapping documents
- Interaction with multiple development teams in their development and enhancement activities
- Distribute the work among team members and follow up with them to make sure that it meets the timeline
- Involve in the design and development of ETL jobs for the processing of data from SAP data source to Redshift database target tables using Informatica listener transformation using source for bci mappings
- Involve in the review of ETL jobs and Unit test document before migrating it in the higher environments
- Involve in the production migration of ETL jobs and smoke testing
- Design and implement log capture feature for each layer which help in production support
- Design the job to process delta records for process improvements
- Design and implement creating parameter files dynamically for ETL jobs which helps in maintenance and support
- Follow various ways to tune of the performance of Informatica jobs on Redshift database
- Define and follow the standard and best practices while developing and reviewing of ETL jobs
- Using Batch Id concept to keep data lineage feature in ETL jobs
- Design and develop Unix shell script for file handling and sftp files to another server
- Provide production support of the various ETL jobs to make sure that it should not impact the business
Environment: Informatica PC 9.6.1, Amazon Redshift DB 1.0, Aginity Workbench, MS Visio 2010, Unix Shell Script, Windows 2012, AWS Cloud
Confidential, New York City, NY
Senior ETL Consultant and Tech Lead
Responsibilities:
- Requirement gathering with various business users by conducting meetings
- Discuss with Business Analyst team to review and finalize the requirement document
- Preparation of high level design and follow up with higher management to get sign off
- Design and development of ETL jobs for the processing of marketing data from Oracle DB to Teradata DB and flat file to Teradata DB
- Design layer wise implementation in the stored procedures to keep restart- ability, maintainability and ease of use features
- Design and implement log capture feature for each layer which help in production support
- Design the job to process delta records for process improvements
- Define and follow the standard and best practices while developing the jobs
- Using Batch Id concept to keep data lineage feature in both ETL and Store Procedures jobs
- Distribute the work among team members and follow up with them to make sure that it meets the timeline
- Provide guidance in the development of various stored procedures in Teradata DB for the processing of data to make it available for Campaign team
- Using BTEQ, Fast Export, Fast Load, Multi Load Teradata utilities to export and load data to and from different source systems
- Design and develop Unix shell script for file handling and sftp files to another server
- Provide production support of the various ETL jobs to make sure that it should not impact the business
- Provide pictorial representation of several existing jobs for better understanding by non-technical person, support team and new comers
Environment: Informatica PC 9.5, Teradata 14, Teradata SQL Assistant, MS Visio 2010, Unix Shell Script, Windows 7, Tidal Scheduler
Confidential, Morristown, NJ
Senior ETL Consultant and Tech Lead
Responsibilities:
- Planned for the upgrade of the ETL jobs from Ver8 to Ver9 in Agile Methodology
- Designed the Data Marts based on the requirements and plan for its load through ETL job
- Redesigned the several existing ETL jobs to make it more secure and automat it where manual effort was applying from long time
- Designed the new ETL jobs by keeping restartability, error handling, data security, ETL standards features and make sure that it is performance tuned
- Distributed the work among offshore team members and make sure that the delivery of the work is in time
- Helped the offshore/onsite team for every functional and technical issues
- Reviewed the work of the team before upgrading the ETL jobs to V9 server
- Scheduled daily Scrum meeting with team to get update on daily basis
- Coordinated with client for any functional clarification as well as help them in data analysis
- Coordinated with Infrastructure team for the infrastructure changes
- Provided the production support for the ETL jobs in EST time
- Provided presentation to the client for several options for process improvement
- Implemented Web Services Informatica job using web service consumer transformation
- Identified the ETL jobs for decommission and help the client for cleanup of several related components like database tables, scheduler job, mainframe jobs, File movement jobs
- Implemented Development, Analysis and Support of PLSQL Stored Procedures
- Documented and Implemented best practices of ETL jobs in Confluence ( Jira tool )
Environment: Informatica PC 9.5, Oracle 11g, PL/SQL, MS Visio 2010, Windows 7, Jira 6.1.1, Autosys Scheduler
Confidential, Newark, NJ
ETL Architect and Tech Lead
Responsibilities:
- Architectural design of ETL(Extraction Transformation and Loading) model
- Implemented ABC (Audit Balance and Control) model in ETL Architecture
- Requirement gathering from various business stake holders
- Designed the different ETL batches to load data from various data sources (flat file, relational, xml) to IDS databases tables
- Involved in the design of the batches to load the data to MDM (Master Data Management) database and also extract the data from MDM to IDS database
- Involved in the Data Modeling for review and to accommodate proposed business changes
- Reviewed Business Requirement Document, Technical Requirement Document, High Level Design Document, Low Level Design Document to make sure all are aligned to each other and meet the business requirement
- Reviewed Source to Target mapping documents prepared by development team to make sure all the source to target transformations are defined properly
- Interacted with development team for any technical or business questions
- Implemented the process of Data Discovery, Data Profiling and Data Cleansing using IDQ tool Informatica Developer before it will go for business transformation
- Reviewed the code of Informatica jobs, UNIX Scripts, SQL Queries to make sure it is following the loading design and following all the standards along with Error Handling and Restartability features
- Implement CDC mechanism on ETL jobs to select only the incremental data from source in real time using Power Exchange
- Reviewed and Implemented of PLSQL Store Procedure to load Data Access Layer
- Designed and Implemented Reusable ETL components and SQL components
- Recommended for improvement in the code to prevent performance issue in the jobs
- Designed the jobs in a way to keep scalability features
- Performance tuning of the jobs using Table partition, Informatica session partition and tuning several Informatica transformations
- Automate the review of hardcoding and standards of the Informatica jobs
Environment: Informatica PC 9.1, Informatica Developer 9.1, Informatica PWX 9.1, Informatica MDM 9.1, Oracle 11g, PL/SQL Stored Procedure, Sql Server 2012, UNIX Shell Scripting, Tivoli Scheduler, Share Point
Confidential, Minneapolis, MN
ETL Architect and Tech Lead
Responsibilities:
- Requirement gathering and Functional Discussion with Business Users
- Closely worked with Operation Architect Group to design the Technical Specification
- Gathered the GAPs during Technical Review Session and made sure that everything document and signed off properly
- Involved in the data analysis and profiling to make sure the ETL design is covering all the data scenarios
- Designed and Implemented reusable mapplets using Informatica Developer tool for cleansing the data like removing spaces, remove duplicates, remove junk characters and standardize data like address, identity etc to maintain data quality
- Implemented CDC (Change Data Capture) mechanism in the ETL job
- Clearly defined the Scope and Assumption of Interfaces, Communicated the Risk and Mitigation plans proactively to Business Stakeholders
- Involved in the extension and changes of the data model of Data Warehouse
- Prepared of High Level Design of Source Data Analysis, ETL Process flow, Error Handling, Restartability, and Extendibility
- Handled the 6 members of development and support team
- Helped the team in Development, mainly on Performance tuning and making sure that all mappings are developed in optimized way
- Distributed the work among the team members to make sure the project delivery is on time
- Interacted daily with team members for their technical and functional questions
- Reviewed data mapping, design documents, Informatica code, Unit Test Cases
- Reported defects and status to the Management and Stakeholders in time
- Worked on the Development and Administration of reports in QlikView tool
- Worked on the Development and support of the Universe in Business Object tool
Environment: Informatica PowerCenter 8.6, Oracle 11g, SQL Server 2005, UNIX Shell Scripting, MS Visio 2003, QlikView 9.0, Business Object 5.1, Control-M Scheduler
Confidential
ETL Architect and Tech Lead
Responsibilities:
- Analyzed and understood business and customer requirements by interacting with Business Analysts, Data Modelers and Subject Matter Experts
- Prepared and reviewed documents like Functional Specifications, Data Mapping Sheets and Technical Specification Documents
- Prepared documents like Deployment Doc, Estimation Reports, Development tracking report and weekly status Reports .
- Established standard code migration process
- Preparation of High Level Design for the ETL jobs with the understanding of Technical Requirement Document
- Handling the 5 members ETL team and distribution of the work among team members
- Involved in the data modeling of Data Warehouse and provided the key changes based on the ETL implementation
- Technical and Functional interaction with team members and helping in their questions
- Mentoring the team in the development of batches using Informatica, SQL and UNIX
- Involved in the implementation of critical processes and provided extra hours support whenever required
- Performance tuning of targets, sources, mappings and sessions based on the bottleneck
- Data investigation in the analysis of incoming data from the various source systems, documenting the data anomalies and generating Data Quality reports
- In addition to developing new mappings / ETL jobs, also responsible for automation, monitoring and supporting mappings, and re-designing if required to make them run faster and more reliable
- Prepared and Implemented UTC (Unit Test Cases) documents
- Developed and provided support of PL/SQL Stored Procedures for the implementation of project
Environment: Informatica Power Center 8.1.0, Oracle 10g, PLSQL, UNIX shell scripts, Share Point
Confidential, Woonsocket, RI
ETL Senior Developer and Team Lead
Responsibilities:
- Work with business users directly in gathering requirement and implementing them and working close with offshore developers
- Prepared user requirement documentation for mapping and additional functionality
- Interpreted logical and physical data models for business users to determine common data definitions and establish referential integrity of the system
- Analyzed the requirements and framed the business logic for the ETL process
- Assisted in the creation and implementation of the business rules via PLSQL Stored Procedures using implicit/explicit cursors, array and using various functions
- Extensively used ETL to load data using PowerCenter from source systems like XML Files, Flat Files, and CSV Files into staging tables and load the data into the target database (Oracle)
- Developed mappings to load the data from various sources into the Data Warehouse, using different transformations like Source Qualifier, Expression, Lookup, aggregate, Update Strategy, Joiner, Normalizer transformations
- Developed slowly changing dimension mappings (Type 1 and Type 2)
- Worked on populating the EDW / Data Marts, multiple fact and dimension tables with star methodology
- Designed and developed reusable mapplets and used mapping variable and mapping parameters in the ETL mapping
- Optimized the performance of the mappings by various tests on sources, targets and transformations, Identified the different bottlenecks
- Involved in documenting the deployment plan and the unit test plan and other UAT related documents
- Did unit testing to identify what can be done to improve effectiveness and efficiency
- Support the User Acceptance Testing (UAT) to see that the system under test meets the user’s needs
Environment: Informatica Power Center 8.1, Oracle 10g, SQL, PLSQL, UNIX shell scripts, Control-M Scheduler, SVN version control
Confidential
ETL Developer
Responsibilities:
- Involved in understanding requirements, analyze new and current systems to quickly identify required sources and targets
- Involved in preparation of technical documentation, design transformations that are consistent with the goals of the existing data warehouse, and worked with the development team to implement the solution
- Worked on legacy code to analyze problems, confirm requirements, and create designs, code and testing.
- Experience in designing, developing, and testing Informatica extract/transform/load processes in a Data Warehouse.
- Extensively worked on Mapping Parameters, Workflow Variables, Mapping Variables, and Session Parameters.
- Used the Workflow Manager to create Workflows, Worklets and Tasks.
- Wrote SQL over ride queries in the required transformations.
- Used design and development techniques that resulted in efficient, maintainable, and high quality ETL processes.
- Used problem solving skills and quickly understood relationships between data with little documentation.
- Developed and Implemented the PLSQL Stored Procedure and provided support in production
Environment: Oracle 9i, TOAD, Informatica 7.1, Unix Shell Scripting, PL/SQL, SVN version control
