Sr. Datastage Developer Resume
eSUMMARY
- 8 years of IT experience focusing in Analysis, Design and Development of ETL processes in all phases of the Data Warehousing Life Cycle with proficiency in Decision Support Systems and OLAP technology.
- 8 years of strong understanding of Data Warehousing tools including IBM DataStage V8.x /Ascential DataStage V7.5 (Designer, Director Manager and Administrator).
- Proven Experience in Data Mapping. Built Source to Confidential Mapping (STM) documents for various projects which involve multiple data sources from different databases including SAP BW, DB2, Teradata, UDB DB2, Oracle, SQL Server, Mainframe DB, XML and web services.
- Hands on experience in fine tuning, tuning performance, identifying and fixing bottlenecks at various levels such as source, Confidential and mapping.
- Worked in National Language Support (NLS) - an additional feature in DataStage which is used to accept data in any character set into most DataStage fields and to process data in a wide range of languages.
- Developed and executed DataStage jobs including Job execution status, monitoring a job, job log view, job scheduling, creating batches and scheduling batches.
- Well-versed in all phases of a project life cycle including Requirements gathering, Analysis and design, Development and coding, Unit testing, Integration testing, Implementation and Maintenance.
- Well-informed on Dimensional Modeling.
- Mastered data warehouse architecture and designing Star schema, Snowflake schema, Fact and Dimensional Tables, Physical and Logical Data modeling
- Proven Capability in dealing with different parallel stages including Join, Merge, Lookup, Filter, Aggregator, Modify, Copy, Sort, Funnel, Change Capture, Change Apply, Remove Duplicates, Surrogate Key Generator, Row Generator, Column Generator and Peek
- Mastered design and development of reports using MicroStrategy & Web FOCUS.
- Strengtaned Experience in UNIX Shell scripting (Bourne Shell (SH), Korn Shell (KSH), BASH, CSH and Cron tab) as part of triggering DataStage jobs, automated scripts, file manipulation, count matching, Scheduling and text processing.
PROFESSIONAL EXPERIENCE
Confidential
Sr. DataStage Developer
Responsibilities:
- Proficient in Analysis, Design, Support and Development of all the different modules in DataStage.
- Designed DataStage jobs& UNIX scripts to handle Key validation, Business code validation, Data cleansing, Surrogate key generation, Landing, Staging, Fact load, Merging the data and preparing the aggregates tables.
- Designed Parallel jobs with DataStage Designer using various active stages like Aggregator, Copy, Join, Lookup, Remove Duplicates, FTP stage, Filter, Lookup, Modify, Transformer, Change Capture, Remove Duplicates, Surrogate key Generator, Sort and Funnel.
- Designed Data Stage parallel jobs to process more TEMPthan 200 million records handled on a weekly basis.
- Extensively used DataStage Director for monitoring and debugging of jobs and sequences.
- Well-versed in the preparation of technical specification for the development of DataStage Extraction, Transformation and Loading (ETL) mappings to load data into various tables in Data Marts and defining ETL standards.
- Experience working with the Performance Mart layer - Performance layer includes Base tables in Dimensional form, at the lowest grain, required for reporting.
- Proven capability to fine tune table objects by selecting accurate Primary Indexes, Secondary Indexes, Partitioned Primary Indexes, and Join Indexes in warehouse.
- Proven experience of data extraction from heterogeneous sources like SAP BW using open hub connection, Mainframe tables, Flat files, IBM DB2, SQL server and Lawson Oracle tables.
- Proficient in using IBM Managed File Transfer (MFT) to share and exchange sensitive files, to protect the transfer of intellectual property, ensure compliance, boost productivity and reduce costs.
- Extracted data from text files, using FTP Stage as well as FTP, SFTP using UNIX scripts and loaded into DB2 UDB and Teradata databases.
- Proven experience and understanding of Agile Methodology approach for projects
- Designed ETL processes that extract information from data sources and populate Enterprise Data Warehouse for Finance Business Intelligence (FBI).
- Developed ELT Scripts/ DataStage Jobs sourcing the incremental data for reports.
- Analyzed data with discrepancies through Error files and Log files for further data processing and cleansing.
- Leading member for technical discussions with architects on Teradata, DB2 & ETL
- Conducted full integration tests and code reviews for all jobs within each sequence before transferring the jobs and sequencers to and from the Development environment and the QA environment.
- Performed code reviews for the developed jobs and fixed the identified defects in the jobs.
- Participated in reviews of data modeling and business requirement analysis and assisted with defining requirements.
- Collaborated with Business users and Subject Matter Experts (SMEs) to understand the business requirements.
- As part of the project team, created Micro Strategy and Web Focus objects on top of Performance Layer and/or Foundation Layer.
- As part of the project set up in Teradata, gained experience performing the following activities.
- Control-M flow and the dependencies setup.
- Teradata user ID setup.
- Performed bulk data migration from DB2 to Teradata through various Teradata utilities like Fastload, MultiLoad, TPump & FastExport.
- Well-versed in Mainframe DB2, JCL, CA7 scheduler, COBOL programming
Environment: Data stage 8.x, Data stage 7.x, Teradata, UDB DB2 9.X, SQL Server 2008, Oracle 11g, Shell script, SQL Developer, Micro strategy, Web FOCUS, SAP-BW, Performance Tuning, Control-M, Mainframe, Visio, Project planning & implementation, Maintenance.
Confidential, CO
Data warehouse Developer
Responsibilities:
- Analyzed, designed, developed, implemented and maintained Parallel jobs using Enterprise Edition of DataStage.
- Involved in design of dimensional database - Star schema and creation of physical tables in Oracle.
- Extracted data from disparate sources - relational databases, oracle database, flat files, XML documents, and loaded into data warehouse.
- Successfully implemented pipeline and partitioning parallelism techniques and ensured load balancing of data.
- Deployed different partitioning methods like Hash by field, Round Robin, Entire, Modulus, and Range for bulk data loading and for performance boost.
- Repartitioned job flow by determining DataStage PX best available resource consumption.
- Experienced in PX file stages that include Complex Flat File stage, Dataset stage, Lookup File Stage, Sequential file stage.
- Implemented various Shared container for multiple jobs and Local containers for same job as per requirements.
- Implemented multi-node declaration using configuration file (APT Config File) for performance enhancement.
- Experienced in developing parallel jobs using various Development/debug stages (Peek stage, Head & Tail Stage, Row generator stage, Column generator stage, Sample Stage) and processing stages (Aggregator, Change Capture, Change Apply, Filter, Sort & Merge, Funnel, Remove Duplicate Stage)
- Worked within a team to populate Type me and Type II slowly changing dimension tables from several operational source files
- Created sequencers to sequentially execute the designed jobs. Used Execute Command Activity Stage, Notification Activity Stage, STP stage, job activity, routine activity and sequencer stages to implement these sequencers.
- Used Data Stage Manager to import metadata, new job categories and creating new data elements as well as to export job categories.
- Created several general routines (Before-After, Transform function) used across the project.
- Used Siebel Pack to extract data from Siebel components and loaded into the data warehouse.
- Involved in creating UNIX shell scripts for database connectivity and executing queries in parallel job execution.
- Used the ETL Data Stage Director to schedule running the solution, testing and debugging its components & monitor the resulting executable versions.
- Interacted with the reports developers to validate the developed data warehouse and created views to facilitate the complexity in developing the reports.
Environment: IBM WebSphere DataStage Enterprise Edition 8.1,IBM MDM, SQL, PL/SQL, Oracle 10g/9i, TOAD 8.5, Unix Crontab Scheduler, UNIX Shell Scripts, Erwin 4.1
Confidential
DataStage Developer & Design Analyst
Responsibilities:
- Implemented and wrote ETL programs to load Customer, Contract, line of Business and Product dimension tables. Also contributed efforts to develop ETL process for Time, Location dimension tables. Worked on Complex ETL programs to implement multilevel hierarchy in dimensional tables.
- Extensively worked with Manager, Designer, Director, Administrator to define the Source and Confidential objects. Extracted Source Data using DataStage tools and stored procedures from Source systems.
- Worked onTeradatadatabase usedTeradataAPI stage with user defined SQL,TeradataEnterprise stage andTeradataMulti-Load stage.
- Configured Oracle, DB2inDataStagefor accessing and reading from database whereas configured TeradatainDataStagefor accessing and writing to database.
- Involved in Importing Metadata from Oracle, DB2 Databases. Used CDC Stage and CA to load Type 2 Dimensional Tables. Used Data Stage Designer to develop DataStage jobs, scheduled the jobs through DataStage director. Tuning of Data Stage Jobs for better query performance.
- Extensively used various stages like Aggregator stage, Hash file stage, Link Collector/Partitioner stage and FTP Stage. Involved in developing Shared Containers for Server Jobs.
- Involved in data stage mapping, data profiling and batch processing.
- Worked Extensively on DataStage Director for scheduling the Job to run, emailing production support for troubleshooting from LOG Files.
- Designed and Developed scripts (shell, AWK, SED) to come up with a generic way to load the data into the table on a particular database.
- Seamlessly migrated the code from Development Testing UAT Production.
- Developed SQL, PL/SQL Stored procedures, triggers and cursors for implementing Complex business rules and transformations and loading.
- Developed UNIX scripts using K-Shell, AWK, SED & UNIX commands to perform net change (incremental data). Developed UNIX Scripts to develop Driver programs to run DataStage Job Control Sequence. Used DS commands to run the DataStage Job Control Sequence.
- Generated completion messages, status reports using capabilities Job Control Sequence and UNIX.
- Responsible for UNIT, System and Integration testing. Developed Test scripts, Test plan and Test Data. Participated in UAT (User Acceptance Testing).
- Scheduled batch jobs in AutoSys. Coordinated with System Operators to Schedule the batch jobs in AutoSys. Provided Level 3-production support.
Environment: Ascential DataStage 8.x, 7.x, PL/SQL, Teradata, SQL, SQL Server, SSIS, UDB DB2, Teradata, Windows NT 4.0, UNIX.
Confidential
DataStage Developer
Responsibilities:
- Involved in various phases of the Data warehousing life cycle like logical and physical design, ETL process, performance tuning, and maintenance.
- Involved in generating Star Schema with facts and dimensions for developing a Datamart in the Data warehouse.
- Extracted data from various relational databases, application systems, flat files, XML documents, and loaded into data warehouse.
- Designed and developed DataStage Parallel jobs and performed data loads.
- Developed parallel jobs using various stages including dataset stage, fileset stage, lookup fileset stage, transformer stage, peek stage, join stage and merge stage
- Manipulated the source system data using copy stage, modify stage, filter stage, join stage and aggregate stage in parallel jobs.
- Profound knowledge on running parallel jobs with multiple configuration file (APT CONFIG FILE)
- Worked with DataStage Manager to import/export metadata, jobs and routines from repository and also created data elements.
- Has done extensive source to Confidential mapping.
- Created local and shared containers to facilitate ease and reuse of the jobs.
- Used the DataStage Administrator to set properties for the DataStage projects to increase the overall performance.
- Performed Unit testing and wrote the test cases for job created.
- Developed Test scripts, Test plan and Test Data. Participated in UAT (User Acceptance Testing).
- Developed Shell scripts to automate file manipulation and data loading procedures.
- To obtain better performance, tuned DataStage jobs by creating DataStage hash files for staging the data.
- Worked on programs for scheduling Data loading and transformations using Data Stage from legacy system and file manipulation and data loading procedures.
- Used explicit cursors for controlling Data Processing.
- Extracted data from Teradata databases.
- Used DataStage Director and runtime engine to schedule running the server jobs.
- Worked with the ETL Architect and technical teams on systems performance and other maintenance issues.
- Documented the Purpose of mapping so as to facilitate the personnel to understand the process and incorporate the changes as and when necessary
Environment: Ascential DataStage 7.5.2 PX, Teradata V2R11, Bteq, SAP R/3 Pack 5.1, Oracle 9i, MS Visio Standard 2002, SQL, PL/SQL, Shell Scripts, and UNIX
Confidential, Atlanta, GA
ETL Developer
Responsibilities:
- Involved in the Analysis of Physical Data Model for ETL mapping and the process flow diagrams for all the business functions and involved in designing the procedures for getting the data from all systems to Data Warehousing system.
- Involved in designing the procedures using Ascential Integrity for data cleansing, used pre-built procedures for cleansing the address data of customer for internal business analytical purposes, the data was standardized to store various business units in tables.
- Distributed the data load to different available processors to speed up the job on Orchestrate environment.
- Implemented extracting, cleansing, transforming, integrating and loading data into Data warehouse using DataStage Designer, used DataStage Manager for importing metadata from repository, new job categories and creating new data elements, used the DataStage Director and the runtime engine to schedule running the solution, testing and debugging its components and monitoring the resulting executable versions (on adhoc or scheduled basis).
- Involved in Installing the Business Objects, creating repository, users and user groups.
- Designing and developing universes for report generation using BO Designer.
- Implemented SOA (Service oriented Architecture) using Datastage as ETL tool.
- Developed user defined Routines and transformations to implement business logic and Shell scripts to automate file manipulation and data loading procedures.
Environment: DataStage 7.5, Oracle 10g, DB2 UDB 7.0, Informix, MS SQL Server 2000, MS Excel 2000, ERWIN 3.4/4.0, Business Objects 5.0, WEBI, SQL Navigator, Syncsort, SQL * Loader, Unix AIX 4.3, Control-m and Windows NT 4.0.
Confidential
ETL Developer
Responsibilities:
- Involved in composition of functional specification/high level design documents based on customer requirements.
- Close work with different teams assigned to different functional areas.
- Validation of designs with client through interviews and design walkthroughs.
- Interaction with clients in remote locations to verify customer requirements.
- Wrote embedded SQL statements and PL/SQL stored procedure calls shared across modules.
- Embedded SQL statements were parsed by Pro*C during the build process.
- Creation of UNIX shell scripts to implement various system/environment administration tasks.
- Development of number of screens using COBOL.
- Enhancement and modifying existing modules.
- Assisted in a conversion of OS/390 platform involving significant JCL modifications.
Environment: Oracle 7.0 & Developer 2000, SQL, D2K, PL/SQL, UNIX, COBOL