Sr. Datastage Developer Resume
Sacramento, CA
SUMMARY
- 8+ years of IT experience in the Development, Implementation and Testing of Database/Data warehousing applications for Financial industries using Data Extraction, Data Transformation, Data Loading and Data Analysis.
- Proficient Data Warehousing with Ascential Data Stage, Quality Stage, Profile Stage and Audit Stage.
- Hands on experience in Data Stage Administration.
- Experience in integration of various data sources like XML, Mainframe COBOL Files, Flat Files, Oracle, SQL Server, Teradata, and DB2UDB EEE into Data Warehouse.
- Extensive experience in loading high volume data and Performance tuning.
- Hands on experience with Data Stage Client Components - Designer, Director and Manager.
- Experience with UNIX shell scripting for File validation.
- Usage of Teradata Bulk Load stages to Load the data into Teradata Database.
- Experience in designing and developing complex Data Stage jobs, routines and sequencers.
- Hands on experience in writing, testing and implementation of the triggers, procedures, functions at Database level using PL/SQL.
- Worked on different scheduling tools like Autosys and Control-M
- Experience in 24/7 production support for various projects.
- Experienced with all phases of software development life cycle. Involved in Business Analysis, Design, Development, Implementation and Support of software applications.
- Following up deployment process of Data Stage code migration on different environments (Development, Test and Production) with admin team.
- Involved in design and implementation of Data Warehouses fact and dimensional tables (Star Schema) with identification of Measures and Hierarchies.
- Developed complex store procedures using input/output parameters, cursors, views, triggers and complex queries using temp tables and joins
- Highly adaptive to a team environment and proven ability to work in a fast paced teaming environment with excellent communication skills.
TECHNICAL SKILLS
ETL Tools: DataStage v8.7 / v8.5/ v7.5.1
Database: Teradata 14/13.10/12, DB2 UDB 9.X/8.X, Oracle 11g/10gMainframe DB2, SQL Server 2005/2008
Database Tools: Teradata SQL assistant, DB2 control centerDB2 command editor, SQL Developer, IBM datastationAdvanced Query Tool (AQT)
Reporting Tools: Microstrategy and Webfocus
Office Tools: MS Office, MS Access, MS Visio, MS one note and MS Project
Other Tools: XML, Web services, IBM MFT, SAP BEx analyzer, JCL, CA7, MSTFS, TADControl-M, Autosys, Service now, HPSD, Remedy and Erwin
Operating Systems: Windows 98/2000/XP/Vista/7, UNIX, AIX
PROFESSIONAL EXPERIENCE
Confidential, Sacramento, CA
Sr. Datastage Developer
Responsibilities:
- Responsible for all technical and system specifications documents for all ETL processes and perform unit tests on all processes and prepare required programs and scripts.
- Analyze and interpret all complex data on all target systems and analyze and provide resolutions to all data issues and coordinate with data analyst to validate all requirements.
- Monitor all business requirements and validate all designs and schedule all ETL processes and prepare documents for all data flow diagrams.
- Perform root cause analysis on all processes and resolve all production issues and validate all data and perform routine tests on databases and provide support to all ETL applications.
- Oversees basic and advanced transformation activities such as normalization, cleansing, aggregation, summarizing, and integration in complex ETL.
- Worked on agile methodology to develop, test and deploy ETL.
- Involved in analysis of source systems, business requirements and identification of business rules.
- Experience in parsing and shredding the XML messages using XML input stage.
- Data Stage job design, development & debugging to populate data to the target tables.
- Created and used different parameter sets and environmental variables in development of DataStage jobs.
- Scheduled jobs using IBM Tivoli Scheduler and used it for testing and debugging the components, and monitoring the resulting executable versions on an ad hoc or scheduled basis.
- Configured multiple nodes and used parallel engine capacity efficiently and developed master sequences to run multiple jobs in parallel.
- Facilitated use of same job to run multiple times by changing the job parameters using Multi-Instance option in source stages.
- Testing of code on local and development environment.
- Responsible for monitoring scheduled, running, completed and aborted jobs.
- Debugging the logic and other connectivity related issues with aborted jobs.
- Worked on QualityStage to verify US addresses using AVI stage.
- Worked on database connections, SQL joins, views, aggregate conditions and hierarchies.
- Responsible for performance tuning by editing SQL queries to enhance the performance of the databases Involved in UAT.
Environment: IBM InfoSphere DataStage v9.1, IBM Quality Stage 9.1, Information Server suite product, IBM Tivoli Scheduler, ClearQuest, ClearCase, IBM DB2 9.7, IBM AIX 6.1.
Confidential - St. Louis, MO
ETL/Data Stage Developer
Responsibilities:
- Working with Business Analyst to understand and get business requirements.
- Implement business requirements into flow diagram, explaining step by step how these business requirements will be implemented into data stage.
- Updated the current flow diagram based on changes that occurred during development process and was involved into updating/adding new steps at flow diagram
- Created Unit Test Cases and Integration Test Plan documents.
- Developed many Parallel Jobs using various Development:
- Processing Stages (Transformer, Aggregator, Change Capture, Change Apply, Join, Lookup, Modify, Filter, Sort, Funnel, Remove Duplicate Stage, Copy, Surrogate stage, XML stage).
- Debug Stages (Peek Stage, Head and Tail Stage, Row generator stage, Column generator stage)
- Restructure (Column Import stage).
- Database Stages (ODBC Connector, DB2 Connector).
- Extract the data from Text files, Transform the data according to the business requirements and load them into Teradata data warehouse.
- Extract data from Teradata Enterprise data warehouse tables and apply business requirements and Outbound the file to xferout folder, ready to be FTP to Vendor
- Creating fix length or Delimited Outbound files.
- Implementing Performance tuning of Data stage jobs with proper data stage partitioning techniques.
- Migration data stage 8.5 jobs to data stage 9.1 version, working with Data Stage Administrator to move, renames or deletes projects and deploying jobs from development to Test stage or from Test environment to Pre-Prod environment and from Pre-Prod to production.
- Creating Test Cases and Performed Unit Testing, End to end System Integration Testing, UAT Testing.
- Wrote Teradata SQL complex queries extracting data using tuning queries and "Explain" statement for better performance.
- Used Data Stage Director for Executing, Monitoring, and Analyzing logs.
- Use Zena as a scheduling tool to handle event based scheduling and time based scheduling.
- Edit Shell Scripts to fit our data stage job like email script, audit script, move file to archive etc.
- Involved in the Production On call Support during Migration event and production deployment.
Environment: IBM InfoSphere DataStage 8.5, 9.1, Teradata SQL Assistance 14.10, IMB DB2 9.7, MQ Visual Edit, IBM AIX 6.1, WinScp, Microsoft-Visio, Serena Dimension, ASG Zena Scheduling Tool.
Confidential - PA
ETL Data Stage Developer
Responsibilities:
- Participating on Business Analysis Development Phase and Gathering Requirements. Working with Developer Department team for translating business requirements into Data Mart Design.
- Working with Database team for creating logical and physical Data Modeling process using Erwin and guide business group and users session.
- Taking part in planning and management of all Data Warehouse migration process.
- Involved in creating strategy for Star Schemas with Fact and Dimension Tables.
- Translating Business requirements into Data Mart design coordinating with team members Creating Fact, Dimensional and Aggregate Tables and Loading Data Warehouse tables.
- Responsible for getting the Data Dictionary, Designing and modifying the Data Stage jobs using Data Stage Designer, Created mappings based on Data Dictionary.
- Worked on Data migration from SQL Server 2005 and was responsible developing Data Stage Jobs, creating mapping for data transformation, extracting data from Oracle and Loading it into the Dimensional tables.
- Creating backups by using Data Stage Manager Exporting data stage components and packaging of projects.
- Implemented the Surrogate Key by using Key Management functionality for newly inserted rows in Data Warehouse
- Developed Shell Scripts for taking backup and recovery of database. Performed physical and logical backup.
- Used Autosys to schedule the Data stage ETL batch jobs on daily, weekly and monthly basis.
- Used Data Stage manager to import, create and edit the metadata.
- Used the Data Stage Administrator to assign privileges to users or users groups, move, rename or delete projects and manager or publish jobs from development to production status.
- Validated all applications and ran the jobs using Data Stage Director.
- Performed tuning of the repository and jobs for optimum performance.
- Prepared Test Plan and performed Unit Testing, System, Integration, Volume Testing.
- Extensively involved in writing Stored Procedures and calling the same through Data Stage Stored Procedure Transformation.
- Developed Server side functionality by using PL/SQL and UNIX shell programming.
- Constructed SQL Scripts to validate the data after loading process.
- Data Quality and validation using Quality Stage, Metadata capture and analysis using MetaStage.
Environment: DataStage EE 7.5/8.1 ERwin 4.1.5, SQLServer 2005, Oracle10g/9i, PL/SQL, AutoSys TOAD, UNIX, Shell Scripting
Confidential - Chicago, IL
Data Stage Developer
Responsibilities:
- Extensively worked with Business and Functional Analysts to get the appropriate Requirements
- Preparing Functional design, Detailed level designs and Mapping documents
- Writing Unit and Integration Test Plan documents
- Developing and maintaining Data Stage Parallel Jobs for 340B Complete Program.
- Coded a set of Data Stage jobs to extract, transform and load data from various sources (Oracle tables with standard and non standard Meta data, flat files) to create an allowed value lookup table.
- Performed code reviews of the developed jobs and fixed the identified defects in the jobs.
- Developed Parallel Jobs using various Development / Processing Stages (Transformer, Aggregator, Change Capture, Change Apply, Join, Lookup, Modify, Filter, Sort and Merge, Funnel, Remove Duplicate Stage) Debug Stages (Peek Stage, Head and Tail Stage, Row generator stage, Column generator stage, Sample stage).
- Created Master sequencers using User variable Activity, Job Activity, Execute Command Stage, Start Loop Activity, End Loop Activity, Nested Condition, Exception Handler Stages
- Created re-usable shared container and local container.
- Implemented the Type 1 and 2 Slowly Changing Dimension.
- Involved in developing Test Cases and Performed Unit Testing, End to end System Integration Testing, UAT Testing.
- Was involved and implemented in Performance tuning of Data stage jobs with proper data stage partitioning techniques and Node configuration
- Added Data Stage Environmental Variables at the Project Level
- Used the Data Stage Administrator to assign privileges to users or user group, move rename or delete projects and manage or publish jobs from development to production status.
- Used Data Stage Director for Executing, Monitoring, Analyzing logs and scheduling the jobs.
- Wrote Shell Scripts to handle pre and post job runs.
- Wrote SQL complex queries for implementing business rules and transformations.
Environment: IBM InfoSphere DataStage 8.1, Oracle 10g, Oracle SQL Developer, Toad, PL/SQL, IBM AIX 6.1, Sun Solaris 10, Microsoft-Visio, Java J2EE, CVS, WinScp
Confidential - Detroit, MI
DataStage Developer
Responsibilities:
- Involved with Business users and ETL Leads from different teams to implement ETL Frame Work using
- DataStage Server/PX combination of jobs.
- Sourced data from various sources like DB2 UDB, Flat Files and CSV files
- Designed jobs using different parallel job stages such as Join, Merge, Lookup, Remove Duplicates, Filter, Dataset, Lookup File Set, Change Data Capture, Switch, Modify, Aggregator, DB2 Enterprise, and DB2 API.
- Involved in developing DataStage Designer- Server and PX jobs for Extracting, Cleansing, Transforming, and Integrating /Loading Data into Data Warehouse
- Developed User Defined subroutines using Universe BASIC to implement some of the complex transformations, date conversions, code validations and calculations using various DataStage supplied functions and routines.
- Developed Job Sequencers with restart capability for the designed jobs using Job Activity, Exec Command, E-Mail Notification Activities and Triggers.
- Extensively designed, developed and implemented Parallel Extender jobs using Parallel Processing (Pipeline and partition parallelism), Restartability techniques to improve job performance while working with bulk data sources.
- Created projects using DataStage Administrator.
- Changed user group assignments.
- Unlocked the jobs from administrator and director.
- Extensively used DataStage Director to Monitor and check the run statistics of the Jobs.
- Extensively used DataStage Manager to Export/import DataStage components.
- Extensively used SQL tuning techniques to improve the database read performance through DataStage Jobs and used Frame Work approach to improve transformation and loading steps.
- Involved in Unit Testing, System Testing, Integration and Performance Testing of the jobs.
- Involved in the Execution and creation of Test Plans Test scripts and job flow Diagrams
- Worked closely with Data Quality Analysts and Business Users for data accuracy and consistency after table loads.
Environment: IBM InfoSphere Information Server 8.1, oracle 10g, DB2 UDB 9.1 Enterprise Edition, Redhat Linux, Autosys 4.5, Connect Direct Putty, Microsoft Visio, Microsoft Project server, Microsoft Portal, Clear Quest,, Microsoft Office(Excel, Word and Power point), Acrobat distiller, Clear Case.
Confidential - Charlotte, NC
Data stage Developer
Responsibilities:
- Analyzed the existing ETL process and came up with an ETL design document that listed the jobs to load, the logic to load and the frequency of load of all the tables.
- Imported data from various transactional data sources residing on SQL Server, DB2 and Flat files and loaded into Oracle database.
- Developed DataStage jobs based on business requirements using various stages like Lookup File, Lookup Stage, Join Stage, Merge Stage and Sort Stage.
- Extensively worked on Data Acquisition and Data Integration of the source data.
- Implemented various process checks, data checks and mail notifications to ensure the quality of the data that is loaded into the data warehouse.
- Created indexes and partitions in the database to improve the performance of ETL jobs.
- Involved in writing shell scripts for reading parameters from files and invoking DataStage jobs.
- Created jobs in ControlM Job Management Scheduler to run the jobs in sequence without manual intervention.
- Created statistical reports using Business Objects.
Environment: Ascential DataStage 7.5, QualityStage 7.5.2, UNIX, Parallel Extender/Enterprise Edition Oracle 9i, DB2, SQL Navigator, Cognos 7.1, XML, SQL, PL/SQL, SQL*Loader, Erwin.
Confidential
DataStage Developer
Responsibilities:
- Participating on Business Analysis Development Phase and Gathering Requirements. Working with Developer Department team for translating business requirements into Data Mart Design.
- Working with Database team for creating logical and physical Data Modeling process using Erwin and guide business group and users session.
- Taking part in planning and management of all Data Warehouse migration process.
- Involved in creating strategy for Star Schemas with Fact and Dimension Tables.
- Translating Business requirements into Data Mart design coordinating with team members Creating Fact, Dimensional and Aggregate Tables and Loading Data Warehouse tables.
- Responsible for getting the Data Dictionary, Designing and modifying the Data Stage jobs using Data Stage Designer, Created mappings based on Data Dictionary.
- Worked on Data migration from SQL Server 2005 and was responsible developing Data Stage Jobs, creating mapping for data transformation, extracting data from Oracle and Loading it into the Dimensional tables.
- Creating backups by using Data Stage Manager Exporting data stage components and packaging of projects.
- Implemented the Surrogate Key by using Key Management functionality for newly inserted rows in Data Warehouse.
- Developed Shell Scripts for taking backup and recovery of database. Performed physical and logical backup.
- Used AutoSys to schedule the Data stage ETL batch jobs on daily, weekly and monthly basis.
- Used Data Stage manager to import, create and edit the metadata.
- Used the Data Stage Administrator to assign privileges to users or users groups, move, rename or delete projects and manager or publish jobs from development to production status.
- Validated all applications and ran the jobs using Data Stage Director.
- Performed tuning of the repository and jobs for optimum performance.
- Prepared Test Plan and performed Unit Testing, System, Integration, Volume Testing.
- Extensively involved in writing Stored Procedures and calling the same through Data Stage Stored Procedure Transformation.
- Developed Server side functionality by using PL/SQL and UNIX shell programming.
- Constructed SQL Scripts to validate the data after loading process.
- Data Quality and validation using Quality Stage, Metadata capture and analysis using MetaStage.
Environment: DataStage EE 7.5, Erwin 4.1.5, SQLServer 2005, Oracle10g/9i, PL/SQL, AutoSys, TOAD, UNIX, Shell Scripting.