Etl Lead Resume
SUMMARY
- Over 11 years of experience in Data Analysis, Design, Development, Testing and Implementation of various Applications using IBM Information server DataStage and Qualitystage
- 7+ years of experience in supporting Predictive analytics and decision support systems
- Worked extensively on integrating and loading various databases like Oracle, Teradata, DB2UDB, MS SQL Server
- Expertise in loading and maintaining Data Warehouses and Data marts using DataStage for ETL processes in Windows and UNIX environments
- Used QualityStage extensively for Data cleansing and standardization of names and addreses
- 5 plus years of experience in developing jobs on Parallel Extender (Enterprise Edition) to achieve better job performance
- Experience in writing UNIX shell Scripts for file validation, email notification and scheduling DataStage jobs via AutoSys.
- Highly experienced in using SQL and Extensive Knowledge in PL/SQL Procedures/Functions and Triggers
- Strong Experience in Debugging designs at job level and good knowledge in reporting tools Business Objects and Cognos
- Expertise in working with xml data and using Real time stages like xml stage, web services and MQ
- Sound experience interacting with clients for gathering requirements and performing analysis and design in achieving the goal of meeting end user requirements.
- Highly motivated professional, technically savvy with proven work ethics, excellent communication, presentation and interpersonal skills
- Expertise in performing root cause analysis and resolving performance bottle necks.
- Dynamic Team player and a fast learner who can quickly adapt to new environments.
TECHNICAL SKILLS
Data Warehousing: Star schema, Snow Flake schema, Slowly Changing Dimensions, OLTP, OLAP, Normalization
ETL: IBM InfoSphere Information Server DataStage and QualityStage 9.1/8.5/8.0/7.5.1/7.1/7.0/6.0 , Parallel Extender (Enterprise Edition)
Data Modeling Tools: Microsoft Visio 2010, Erwin 4.1/3.5.2
Databases: Oracle Exadata X5, Oracle 11g/10g/9i/8i/8x/7x, MS SQL Server 2008/2005, DB2 8.x, MS Access 7.0. Teradata V2R6.0, DB2 on AS400
Programming: SQL, PL/SQL, UNIX Shell Scripting, C, C++, MATLAB, XML.
Operating Systems: HP - UX, IBM AIX 7.1/6.1/5.2/4.2/4.3 , Windows 2010/2007/XP/NT/2000/98 SOLARIS 2.x, LINUX.
Tools: /Utilities MetaData Workbench, Business Glossary, Teradata SQL Assistant, BTEQ, TOAD 11.5, SQL*Plus, SQL*Loader, Oracle Forms 4.5/5.0, Reports 2.5/3.0, MS Access Reports.
PROFESSIONAL EXPERIENCE
Confidential
ETL Lead
Responsibilities:
- Was responsible for the design and implementation of the data integration processes from various source systems into a target database.
- Provided relevant data using webservice to the defined Predictive algorithms and models to mitigate risk.
- Building the ETL architecture and Source to Target mapping to load data into Data warehouse.
- Used Quality Stage extensively to standardize and match the customer names and addresses to load a centralized customer database.
- Defined strategies for data extraction, cleansing and loading, as well as developed jobs needed to perform transformation and loading into the new data warehouse structures.
- Developed Parallel jobs using Message Queue (MQ stage) to process portal requests in Real Time with xml data.
- Worked extensively on parsing and composing XML data using XML stage.
- Developed PL/SQL Procedures for realtime Portal Use.
- Worked rigorously on performance testing, user acceptance testing and regression testing.
- Participated in defining user requirements for analytical information delivery
- Provided the design and technical documentation and visio diagrams for the interface daily load processes.
- Worked rigorously with the BI team to modify existing packages and create new packages and reports when new source systems/data is added.
- Resolved data issues and provided root cause analysis in a timely manner and ensured successful completion of Production batch cycle
- Mentored technical resources, performed code reviews, inspected data development procedures to establish standards.
- Optimized datastage code to enhance the batch cycle performance
Environment: IBM Information Server 9.1/8.5, Oracle Exadata X5, Oracle 11g/10g, SQL server 2008, Erwin4.0, DB2 AS400, Sun Solaris, TOAD, AIX 7.1
Confidential
ETL Lead
Responsibilities:
- Designed high-Level design and process documents for ETL Server and parallel Jobs for the inbound interface.
- Building the ETL Source to Target mapping to load data into Data warehouse.
- Work quickly and independently to resolve issues, provide maintenance and production support.
- Developed data stage server jobs for Policies and Claims data migration and interface data loads.
- Worked with flat files with fixed length and comma delimited files.
- Developed PL/SQL Procedures for building Summarized monthly snapshot tables.
- Extensively worked in Quality Stage to clean up the address information within the client data.
- DevelopedjobsinParallelExtenderusingdifferentstageslikeTransformer, aggregator,lookup, join, merge,Sourcedataset,externalfilter,Rowgenerator, andcolumn generator.
- Developed the ETL jobs using Real-Time Integration stages like Web Services, XML Input/outout and XML transformer stage
- Written several routines to be used in the jobs according to the business rules.
- Extensively developed jobs using FTP stage to send/receive files from external servers.
- Extensively involved in writing the UNIX shell scripts for file validations
- Incorporated SQL Loader script, invocation of PL/SQL procedures and DataStage jobs/sequencers list files to execute in one single invocation shell script using the existing generic shell scripts with project vise DEV,QA and PROD parameterized environment.
- Extensively used SQL queries to evaluate the data in Source and Target Databases
- Created MS visio diagrams for the entire process workflow.
- Involved in the Data Stage version upgrade and moved the existing Inbound and Outbound production objects
Environment: IBM Information Server 8.5/8.1/7.5, Parallel Extender, SQL server management studio 2008, Erwin4.0, Oracle 11g/10g/9i, DB2, Sun Solaris, TOAD
Confidential
Senior DataStage Developer
Responsibilities:
- Participated in the review and approval of the technical transformation requirements document used to design and build the extraction, transformation, and loading (ETL) modules in DataStage Designer
- Participated in Master Data Management Initiation for SAP Module.
- Extensively used DataStage Designer to develop processes for extracting, cleansing, transforming, integrating, and loading data into Intermediate database
- Used Ascential DataStage Manager to Import and Export DS Objects
- Used DataStage Administrator to make Project wide changes such as create user variables, change tunables.
- Created Jobs in DataStage to transfer from heterogeneous data source like CSV files, DB2, SQL server to Oracle
- Written UNIX shell scripts for Data Acquisition as a part of ETL process
- Created Custom Job Batches (DS job control functions) and Sequencers to control the flow of DataStage job invocations
- Created user defined Routines using Job Control functions and Transform functions in the process of designing the jobs
- Used DataStage BAPI Load Stage for Loading Data in SAP and ABAP Extract Stage for extracting Data from SAP System
- Used QualityStage for Master Data Management in maintaining Unique Customer ID.
- Extensively coded user defined SQL for overriding the column generated SQL query in DataStage for extraction, look up and Loading into database
- Used SQL Loader to Load bulk data into tables
- Created PL/SQL procedures, functions, and triggers based on the requirement
- Performed extensive Unit testing and involved in Integration testing, Regression testing and UAT
Environment: IBM Information Server WebSphere DataStage 8.0, QualityStage, Oracle 9i, WINDOWS 2000, UNIX HP UX, TOAD 8.5.
Confidential
Senior Programmer Analyst
Responsibilities:
- Involved in creating Source to Target Mapping documents and Visio diagrams
- Designed & Developed DataStage jobs which involved data cleansing, complex transformations, validation, referential integrity, surrogate keys
- Used Ascential DataStage Designer to develop processes for extracting data from various sources including Teradata, Oracle, fixed width and delimited files, integrating and loading data into data warehouse database.
- Extensively used Teradata multiload, fastload, fastexport and bteq utilities.
- Used Sequential file Stage, Look-up Stage, Join Stage, Dataset stage and Merge Stage to do transformations on the data.
- Used Ascential DataStage Manager for importing metadata from repository, exporting and importing jobs and data elements from/to different projects.
- Used DataStage Director to monitor performance statistics.
- Extensively used SQL coding for overriding for generated SQL query in DataStage.
- Wrote UNIX shell Scripts for file validation, transformation and triggering DataStage jobs.
- Used Job Control routines and Transform functions in the process of designing the job
- Created unit test, system test plan documents and performed unit testing and documented test results.
Environment: Ascential DataStage 7.5 Enterprise Edition (Parallel Extender), Teradata V2R6, Oracle 9i, WINDOWS XP, UNIX HP UX, SQL Assistant, TOAD 8.5.
Confidential
DataStage Developer
Responsibilities:
- Involved in analyzing the scope of application, identifying the relationship between the groups of data.
- Developed and designed new processes and fixed the existing process with new business requirements.
- Used Ascential DataStage Designer to develop processes for extracting, cleansing, transforming, integrating and loading data into data warehouse database.
- Extensively used the sequential file stage, hash file Stage, Oracle OCI stage, Aggregator, Link collector, Link partitioner and other database plug-ins to do transformations and load the data.
- Created Jobs in DataStage to transfer from heterogeneous data source like ANSI X12 (fixed width flat files), CSV files, Oracle, Access to ORACLE 9i.
- Extensively used SQL coding for overriding for generated SQL query in DataStage.
- Extensively involved in creating PL/SQL procedures, functions, and triggers.
- Wrote UNIX shell Scripts for file validation and scheduling DataStage jobs.
- Used Job Control routines and Transform functions in the process of designing the job.
- Used DataStage and SQL*Loader to load the data into target Oracle database.
- Involved in Tuning the Mappings and Sessions for improved performance.
- Debugged complicated jobs to fix data issues.
- Extensive Unit testing and System testing of designed jobs.
Environment: Ascential DataStage 7.5.1 Server Edition (Designer, Manager, Director, Administrator), Oracle 9i/ 8i, WINDOWS 2000, UNIX AIX 5.2, PL/SQL Developer, SQL Loader, TOAD.
Confidential
DataStage Developer
Responsibilities:
- Designed Mappings between sources to operational staging targets - ODS
- Implemented logic for Slowly Changing Dimensions (Type 1 & Type 2).
- Involved in the development of DataStage Jobs to tune them for better performance.
- Developed jobs on Parallel Extender to achieve better job performance and throughput by efficient usage of system resources in Orchestrate environment.
- Developed Custom stages for Parallel Extender like Buildops, using C/C++ programming
- Created Jobs in DataStage to transfer from heterogeneous data source like COBOL, fixed record flat files, CSV files, DB2, Oracle and Text files to Teradata.
- Wrote UNIX shell Scripts for file validation and scheduling DataStage jobs and also in Orchestrate environment.
- Created Batches and Sequencers to control Sequence of Jobs using Job Controls.
- Involved in Unit and Integration testing of designed jobs
- Extensively used Teradata Load and Unload utilities such as Multi Load, Fast Export and Bulk Load stages in Jobs for loading/extracting huge data volumes
- Developed various SQL scripts using Teradata SQL Assistant and used some of them in DS Jobs with BTEQ Utility and used in Teradata Stages as SQL override
Environment: Ascential DataStage 7.0 Server Edition (Designer, Manager, Director), Parallel Extender (Orchestrate), Erwin4.0, Oracle 9i/ 8i, DB2 UDB 8.x, Teradata V2R4, WINDOWS 2000, UNIX AIX 5.2.