Sr.etl Consultant / Admin Resume
Englewood, CO
SUMMARY
- Thirteen years of experience in full lifecycle development (SDLC) including 8+ years of Ascential DataStage design and over 5+ years of experience in Database design, and development in medium to large enterprise Data Warehousing environments
- Worked extensively with Dimensional modeling, data migration, data cleansing, building ETL processes for Data Warehouses
- Expertise in working with Data Warehouse applications, directly responsible for the Extraction, Transformation & Loading of data from multiple sources into Data Warehouse using IBM (formerly Ascential) DataStage Enterprise Edition (Server and Parallel)
- Excellent working and scripting experience of UNIX (Shell Scripting) and experience on Windows, UNIX and LINUX platforms.
- Responsible for installing/upgrading, configuring and administering of IBM (formerly Ascential) DataStage Information Server 8.0.1/7.5.1 on Unix (Solaris, AIX, HP - UX) and on Windows (2000/2003)
- Experienced in logical and physical data modeling, Dimensional Data Modeling and OLTP design
- Experienced in design and implementation of Data warehouse methodology, metadata designs, schema designs like Star schema and Snow flake Schema etc
- Excellent Experience in Performance Tuning, Backup and Recovery process, and product support on various platforms. SQL Tuning and creation of indexes and Partitions for faster database access and better query performance. Experience in creation of Database objects like Stored Procedures, Functions, Packages and Triggers in database languages like PL/SQL
- Expertise in working with various operational sources like SQL Server, DB2, Oracle, Teradata, Flat files into a staging area
- Excellent problem solving, analytical, written and communication skills with ability to work both in team as well as an individual
- Excellent work ethics, self-motivated, quick learner and team oriented
TECHNICAL SKILLS
Data warehousing: IBM WebSphere DataStage 8.5/8.0.1/7.5.1/7.5/6.0 (Administrator, Manager, Designer, Director, Parallel Extender, Quality Stage/Integrity, MetaStage), Ascential QualityStage 6.0, DataStage Plug-In, Data Mining, OLAP and OLTP, Data Mining, Datamart, OLAP, OLTP, SQL*Plus, SQL*Loader, Cognos 6.0/5.x
Data Modeling: Star and Snowflake schema modeling FACT and Dimensions, Physical and Logical Data Modeling.
Databases: Oracle 10g/9i/8i/8.0, PL/SQL, Teradata, UDB DB2, Sybase SQL Server 11.0, MS SQL Server 6.5/7.0, TSQL, MS Access 7.0/97/2000, Excel
Languages: SQL, PL/SQL, UNIX Shell Scripting, Perl, C, Java
Web Tools: Flash, Dream weaver, HTML, XML, Rational Rose and JavaScript
PROFESSIONAL EXPERIENCE
Confidential, Englewood, CO
Sr.ETL Consultant / Admin
Responsibilities:
- Involved in Knowledge Transition Sessions for the Projects to be deployed to Production
- Deploying the Datastage Projects to Production
- Monitoring the jobs on Daily basis and incase of failure resolving the issues
- Performance improvement for the jobs consuming more resources
- Developed and Improved few shell scripts on AIX environment
- Worked on Minor enhancements for the jobs. Extensively used IBM Datastage Designer, Administrator, Director, and Integrity for creating and implementing jobs.
- Experience with Unix shell scripts
- Fine tuning the queries causing issues and improving the performance
- Developing Stored procedures and tweaking the existing ones as per business logic
- As Part of Admin Activities Created new projects
- Created New users and assigning the roles and access to specific projects
- Resolving login issues and log corruption issues to users
- Resolving Space and Memory issues created due to Resource intensive jobs running on the server
- Scheduling the jobs using Datastage Director / Crontab / Autosys tools
- Cataloguing DB2 databases in order to connect to DB2 databases
- Removing locks on Projects
- Involved in several other Admin activities like Recycling the Datastage server resources
Environment: IBM WebSphere Information Server 8.5 (Designer, Director, Administrator), IBM WebSphere Information Server 8.1, Netezza, Oracle 10g, DB2,SQL server, IBM Data Studio, TOAD 8.6, SQL Developer, UNIX-SUNOS 5.10, SHELL SCRIPTING, AIX
Confidential, Grand Rapids, MI
Sr.ETL Developer
Responsibilities:
- Analyze the existing SSIS packages. Designing and modifying some of the existing SSIS packages so that evolving customer requirements are met.
- Redesigning the SSIS packages in DataStage.
- Used DataStage Designer for importing metadata from repository and creating new data elements.
- Worked in the tech specs (Source-Target mappings) for the ETL mappings.
- Involved in designing various DataStage jobs as per requirements.
- Used User-defined SQL Queries to Extract the Required Data elements or attributes from different source systems.
- Used stages like Lookup, Join, Sequential File, Copy, Transformer, Dataset, Peak, Funnel, Row Generator and other stages.
- Used stages like CDC, SCD to handle the changing data.
- Developed Parallel jobs using stages like: Merge, Join, Lookup, Transformer (Parallel), Dataset, Oracle Enterprise Stage.
- Used Quality Stage stages like Investigate, Standardize for Persons Legal name and Address.
- Used Remove Duplicates stage to remove the duplicates in the data.
- Used Filter stage to filter the records.
- Used several stages in Sequencer like Terminator, Wait for file and Email Notification stages to build an overall main Sequence to accomplish Re-start ability.
- Created DataStage jobs, batch jobs and job sequences and tuned them for better performance.
- Performed unit testing of all monitoring jobs manually and monitored the data to see whether the data is matched.
- Used the DataStage Director to schedule the jobs.
- Involved in testing, debugging and monitored the resulting executables.
Environment: IBM WebSphere Information Server 8.5 (Designer, Director, Administrator), SQL Server 2008 R2,Oracle 10g, Business Intelligence development Studio (Visual Studio 2008), Batch file scripting.
Confidential, Troy, MI
Sr. DataStage Developer
Responsibilities:
- Worked on analyzing the systems and gathering of requirements.
- Work with the project and business teams to understand the business processes involved in the solution.
- Involved in designing and development of data warehouse environment.
- Extensively interacted with business Analysts for design of Dimensions and facts tables for data warehouse using Star and Snow Flake Schema.
- Extensively used IBM DataStage Designer, Administrator, Director, and Integrity for creating and implementing jobs.
- Designed and Developed Extract, Transform and Load (ETL) processes from a variety of Transactional systems including legacy systems utilizing SQL, DataStage and Unix shell Scripts.
- Used DataStage Designer to extract, cleanse, transform, integrate, and load data into the data mart.
- Created Oracle PL/SQL scripts to pre-process and post-validate the data in DM
- Extracting data from Oracle, SQL Server, Informix and Flat File sources. Cleansing, transforming and loading data into the target database using DataStage Designer.
- Preparation of technical specification for the development of Extraction, Transformation and Loading (ETL) mappings to load data into various tables in Data Marts and defining ETL standards.
- Created local and shared containers to facilitate ease and reuse of jobs.
- Extensively used most of the transforms of DataStage for various types of transformations.
- Validated and successfully executed jobs using DataStage Director.
- Set up DataStage Design Standards and formulated Unit and Integration Test plans.
- Involved in fine tuning, trouble shooting, bug fixing, defect analysis and enhancement of the multiple DataStage jobs
Environment: IBM WebSphere Information Server 8.5 (Designer, Director, Administrator), IBM WebSphere Information Server 8.1, Netezza, Oracle 10g, TOAD 8.6, SQL, UNIX-SUNOS 5.10, SHELL SCRIPTING
Confidential, Columbus, OH
Sr. ETL Consultant
Responsibilities:
- Delivered a complete roadmap for the project and design of the Data Integration Project using the Ascential Platform
- Did the Gap analysis for the system to build the DataMart, Designed the Logical design of the DM.
- Worked with Business Users and designed reports layouts
- Used DataStage Designer for importing metadata from repository, new job categories and creating new data elements
- Designed/wrote the tech specs (Source-Target mappings) for the ETL mappings along with the Unit Test scripts
- Involved in designing various jobs in PX, DataStage as per given specs
- Used Parallel Extender 6.0/DataStage 8.0 to extract data from source systems such as Oracle, Flat files, XML to target system on Oracle
- Used stages like Lookup, Join, Sequential File, Transformer, DataSet, Peak, Funnel, Row Generator, Row Merger and other stages
- Developed Parallel jobs using Parallel stages like: Merge, Join, LookUp, Transformer (Parallel), Oracle Enterprise Stage
- Performed debugging on these jobs using Peek stage by outputting the data to Job Log or a stage
- Performed testing of these Parallel jobs using Row Generator and Column Generator stages
- Used Remove Duplicates stage in PX (EE) to remove the duplicates in the data
- Used Filter stage to filter the records
- Worked with DS Sever stages like Transformer, IPC, Aggregator, Sort, Link Partitioner, Link Collector and others
- Created UNIX shell script to FTP the flat file from remote third party server to our local server or also used FTP stage for this
- Worked with XML stages for handling the XML data and used Folder Stage to read the XML data
- Involved in massive cleansing and profiling of the production data
- Used Information Analyzer for Profiling the data.
- Created DataStage jobs, batches and job sequences and tuned them for better performance
- Used the DataStage Director and its run-time engine to schedule running the solution, testing and debugging its components, and monitoring the resulting executable versions (on an ad hoc or scheduled basis)
Environment: DataStage EE 8.1 (Designer, Director), DB2, Oracle 9i, TOAD, SQL/PLSQL, Unix (HP-UX) and NT, Shell Scripting
Confidential, Baltimore, MD
Sr. DataStage Consultant
Responsibilities:
- Used the DataStage Designer to develop processes for extracting, cleansing, transforming, integrating, and loading data into data warehouse database.
- Create master controlling sequencer jobs using the DataStage Job Sequence.
- Used DataStage Manager for importing metadata from repository, new job categories and creating new data elements.
- Used Data Stage Parallel Extender parallel jobs for improving the performance of jobs.
- Used Parallel Extender Development/Debugging stages like Row generator, Column Generator, Head, Tail and Peek.
- Dealt extensively with data from Oracle, DB2, TeraData, XML, Flat file and SQL Server sources.
- Used Data Stage Director to validate schedule run and monitor the data stage jobs.
- Developed user defined Routines and Transformations by using Universe Basic.
- Used Before/After Job-Subroutines in Job Properties.
- Responsible for performance tuning
- Extensively wrote SQL script
- Used Data Stage for developing, programs for scheduling data loading and transformations
- Developing shell scripts to automate file manipulation and data loading procedures
- Error Logs/Audit Trails were maintained
- Produced aggregate fact tables for different users in Data Marts
- Involved in the process design documentation of the Data Warehouse Dimensional Upgrades
- Involved in designing the procedures for getting the data from all systems to Data Warehousing system
- PL/SQL procedures to transform data from staging to Data Warehouse Fact and summary tables
- Created various DataStage Hash files for lookups
- Created sessions and batches to run the mappings and set the session parameters to improve the load performance
Environment: IBM WebSphere DataStage 7.5.1, Netezza, Microsoft Analysis Services, Microsoft Visio, Quality Stage, Oracle 9i,Teradata, DB2, SQL, C/C++, UNIX, Windows 2000
Confidential, Boston, MA
DW Developer
Responsibilities:
- Involved in scope definition, requirements gathering, analysis, logical and physical model creation of the database using STAR schema.
- Generated Star Schema - Facts and dimensions for developing Data mart in the Data warehouse.
- Designed and developed DataStage Server jobs and performed data loads.
- Comprehensive expertise on DataStage server components - DataStage Repository, DataStage server, DataStage package installer.
- Dumped the Look-up data into Hash-files and accessed that data using Hash-file stage to drastically improve the performance of the jobs
- Enforced good Developing and Documentation Principles throughout the project.
- Writing routines to schedule batch jobs to obtain data overnight from various locations
- Used ERWIN as leading data modeling tool for Logical (LDM) and physical data model (PDM).
- Processed Cleansing, Purging and Optimizing of the data in warehouse.
- Used Row Generator, Column Generator stages to create test data, Peek stage for debugging.
- Worked with environment variables like: APT CONFIG FILE, APT DUMP SCORE
- Worked with other PX stages like Data Set, Merge, Join, Parallel Transformer, Oracle Enterprise stages and other stages
- Exported the data in the XML format for third party systems
- Worked with XML Input, XML Output stages to export/import the data from other systems
- Used Parallel Extender for splitting the data into subsets and flowing of data concurrently across all available processors to achieve job performance.
- Used DataStage Designer to develop DataStage jobs, scheduled the jobs through DataStage director, and run the jobs in the DataStage Server.
- Working on Multi file systems with extensive parallel processing.
- Involved in writing transforms, routines and stored procedures.
- Design complex job control processes to manage a large job network
- Developed Server Side functionality by using PL/SQL, PERL and UNIX shell programming.
- Developed Korn shell scripts to automate the Data load processes to the target Data warehouse
- Worked closely with the Data Warehousing Admin and Data Modeling team in tuning the Extraction and Summarization process for better performance
Environment: DataStage 7.X (Manager, Designer, Director), DataStage EE, PL/SQL, DB2, DB2 UDB, Erwin, UNIX shell script, Windows NT, UNIX, Oracle 8i (8.1.6), ERwin4.0
Confidential, Chicago, IL
Sr. Data warehouse Developer
Responsibilities:
- Involved in development phase meetings for Business Analysis and Requirements Gathering
- Created DataStage jobs, batches and job sequences and tuned them for better performance
- Developing Architecture for building a Data warehouse
- Extensively used Link Partitioner and Link collector in improving the performance.
- Worked on programs for scheduling Data loading and transformations using DataStage from legacy system to Oracle 8i using SQL* Loader and PL/SQL
- Designing and Developing PL/SQL Procedures, functions and packages to create Summary tables
- Used the DataStage Director and its run-time engine to schedule running the solution, testing and debugging its components, and monitoring the resulting executable versions (on an ad hoc or scheduled basis)
- Worked with DataStage Manager for importing metadata from repository, new job Categories and creating new data elements
- Written shell scripts for Data Acquisition as a part of ETL process
- Involved in jobs and analyzing scope of application, defining relationship within & between groups of data, star schema, etc
- Identifying suitable dimensions and facts for schema
- Creating and loading data warehouse tables like dimensional, fact and aggregate tables using Ascential DataStage
- Used Job Control routines and Transform functions in the process of designing the job
Environment: Ascential DataStage 7.0 (Designer, Director, Manager), Ascential Parallel Extender 6.0, Oracle 8i/9i, Windows 2000, ERwin and UNIX
Confidential
Data Warehouse Developer/ Analyst
Responsibilities:
- Worked with the Business analysts and the DBAs for requirements gathering, business analysis, testing, and metrics and project coordination
- Involved in Dimensional modeling of the Data warehouse and used Erwin to design the business process, dimensions and measured facts
- Created PL/SQL procedures for processing business logic in the database.
- Tuned SQL queries for better performance.
- Created mappings and complex transformations using Ascential DataStage Designer
- Involved in massive cleansing and profiling of the production data
- Involved in scheduling of jobs to be run in batches
Environment: DataStage 6.0, Oracle 8i, Erwin, SQL, PL/SQL, UNIX (HP-UX), Windows NT
Confidential
Data Base/ETL Developer
Responsibilities:
- Involved in the Logical and Physical design of the system using Erwin.
- Designed and developed table structures, PL/SQL stored procedures, functions to implement rules. Modified existing procedure, function queries according to requirements and optimization in PL/SQL queries.
- Designing and tuning of the application to improve the performance.
- Implementing the Partition Tables, Partition Indexes etc. Worked with huge tables with multimillion rows as needed.
- Backup using Export/Import utility, developed database backup and recovery strategies.
- Prepared a production monitoring and support handbook.
- PL/SQL coding at the database level for the new objects.
- Documentation and creation of test cases.
Environment: ORACLE 8i, UNIX, Windows NT, Erwin, Shell Programming, PL/SQL, SQL, Developer 2000, TOAD Quest Software