Sr. Etl Datastage Developer/analyst/quality Stage Developer Resume
Bloomington, MN
PROFESSIONAL SUMMARY:
- Over 8 years of expertise in software design, development and deployment of large and complex software applications.
- Experience in Data Requirement Analysis, Design, Development of ETL process using IBM DataStage 11.X, 8.X/7.x (WebSphere, Ascential DataStage). Expert in all the stages of Software Development Life Cycle (SDLC).
- Experience in Data Warehousing applications, responsible for the Extraction, Transformation and Loading (ETL) of data from multiple sources into Data Warehouse.
- Experience in using various stages like Join, Merge, Lookup, Remove Duplicates, Sort, Filter, Funnel, Dataset, Change Data Capture, Slowly Changing Dimensions, Switch, Modify, and Aggregator in datastage designing.
- Experience in design and development of parallel jobs using Parallel Extender for splitting bulk data into subsets and to dynamically distribute to all available nodes to achieve best Job performance, Involved in maintaining logical and physical models.
- Developed efficient jobs for data extraction/transformation/loading (ETL) from different sources to a target data warehouse.
- Excellent Experience in Designing, Developing, Documenting, Testing of ETL jobs and mappings in Server and Parallel jobs using Data Stage to populate tables in Data Warehouse and Data marts.
- Experience in new enhancements in the IBM Websphere DataStage - Multiple Job Compile, Surrogate key generator Stage, Job Report, Message handler options
- Worked on DataStage production job scheduling process using the Scheduling tools and data stage scheduler.
- In Depth knowledge in building Enterprise Data Warehouses (EDW), Operational Data Store (ODS), Data Marts, and Decision Support Systems (DSS) using Multidimensional Model (Kimball and Inmon) addressing Slowly Changing Dimensions (SCDs).
- Excellent knowledge in Extraction, Cleansing and Modification of data from/to various Data Sources like Flat Files, Sequential files, Comma Delimited files (.csv), XML and Databases like Oracle, ODBC, DB2, Teradata etc
- Extensive experience in writing UNIX shell scripts for data manipulation and transformation.
- Worked on DataStage client tools like DS Designer, DS Director and DS Administrator, Parallel Extender/Orchestrate.
- Detailed knowledge in Data Warehousing concepts like Dimensional Modeling, Star Schema, Snowflake Schema and ETL.
- Used Both Pipeline Parallelism and Partition Parallelism for improving performance.
- Identified and tracked the slowly changing dimensions, heterogeneous Sources and determined the hierarchies in dimensions.
- Worked with SQL, SQL*PLUS, Oracle PL/SQL Stored Procedures, Triggers, SQL queries and loading data into Data Warehouse/Data Marts.
- Experience in using software configuration management tools like Rational Clear case/Borland star team for version control. Skilled in writing technical specification documents, translating user requirements to technical specification.
- Excellent analytical, interpersonal and communication skills with aptitude to assimilate new technologies.
- Excellent experience in Relational database (RDBMS), Oracle 10g/9i/8i, Microsoft SQL Server, Teradata Load, Teradata MultiLoad, SQL, Loader, TOAD.
TECHNICAL SKILLS:
ETL Tools: DataStage 11.X/8.X/7.X (Manager, Designer, Director, Administrator, Parallel Extender), DataStage Plug-In, OLAP and OLTP.
Databases: Oracle 11g/10g/9i/8i, MS SQL Server 2000/7.0/6.5 , DB2 UDB, Hadoop, Teradata, Netteza, MongoDB
Reporting Tools: Business Objects, Crystal Reports.
Data Cleansing: First Logic, Trillium, SQL Scripts, PERL scripting and Unix Scripts.
Data Modeling Tools: Designer 2000, Star Schema, Snowflake Schema.
Operating Systems: Windows NT 98/2000/2003 , UNIX,LINUX MS-DOS.
Programming Languages: SQL, C, C++, C#.
PROFESSIONAL EXPERIENCE:
Confidential, Bloomington, MN
Sr. ETL DataStage Developer/Analyst/Quality Stage Developer
Responsibilities:
- Worked with the Business analysts and the DBA for requirements gathering, analysis, testing, and metrics and project coordination.
- Developed documents like Source to Target mapping for developing the ETL jobs.
- Populated Data Marts at different levels of granularity for the inside Customers using DataStage, SQL scripts and stored procedures (being executed from shell scripts driven by Control-M).
- Imported the required Metadata from heterogeneous sources at the project level.
- Used DataStage Netezza Enterprise stage to load data, utilizing the available processors to achieve job performance, configuration management of system resources in Orchestrate environment.
- Involved in the deployment of DataStage jobs from Development to Production environment.
- Developed Parallel jobs using Parallel stages like: Merge, Join, Lookup, Transformer (Parallel), and Oracle Enterprise Stage, Dataset, Funnel, Change Data Capture and Pivotal.
- Experienced in using advance datastage real time stages like Webservices, XML and used Regroup, parser, join and sort steps in xml.
- Extensively used Lookup, Join, Merge stages for joining various information and also used Parallel Transformer, Column Generator, Funnel, Filter, Switch, Modify, Pivot, Row Generator.
- Used DataStage Designer to create the table definitions for the CSV and flat files, import the table definitions into the repository, import and export the projects, release and package the jobs.
- Performed debugging on these jobs using Peek stage by outputting the data to Job Log or a stage.
- Used DataStage Designer to create the table definitions for the CSV and flat files, import the table definitions into the repository, import and export the projects, release and package the jobs.
- Worked on programs for scheduling Data loading and transformations using DataStage from legacy system to Oracle 10g using SQL* Loader and PL/SQL.
- Designed and implemented slowly changing dimensions (SCD’s) methodologies
- Wrote several complex SQL queries to extensively test the ETL process.
- Developed job sequences to execute a set of jobs with restart ability, check points and implemented proper failure actions. Scheduled various ETL Batch processes using AUTOSYS.
- Created xsds for file parsing for loading into destination.
- Participate in walk-through and provide approval of Test Plan and Test Cases, Participate in defect reviews.
- Created Control-M UNIX job to schedule the jobs to be run the given time and completion of the dependent jobs.
- Responsible for monitoring all the jobs that are running, scheduled, completed and failed. Troubleshooting the failed jobs is a primer in these situations.
Environment: IBM InfoSphere DataStage, DB2, Oracle 11g, Squirrel, SQL Server, Erwin, UNIX, Control-M, Soap UI
Confidential, Little Falls, MN
Sr. Datastage Developer
Responsibilities:
- Effectively and efficiently interacted with the client in gathering business requirements for the ETL module.
- Gathering the metadata definition of the source systems, preparing Transformation rules according to the business requirements for new enhancements.
- Extracted the data from the Oracle database transforming based on business requirements and loading into downstream Mainframe files/Oracle database for various services to run reports.
- Worked on database connections, SQL joins, views, aggregate conditions, parsing of objects and hierarchies.
- Computed extensive transformation logic within the ETL Jobs using the stages like Sequential file, Dataset, Transformer, Copy, Aggregator, Join, Lookup, Shared Container, XML Input, Funnel, Remove Duplicates, Filter.
- Actively involved in everyday job monitoring and resolved all the issues which are related to job aborts and job failures.
- Used Data Stage Director to validate schedule run and monitor the data stage jobs.
- Created error files and log tables, audit tables containing data with discrepancies to analyze and re-process the data.
- Generated XML files by using the XML output stage as a part of Delta Job process.
- Created job schedules in ESP to automate the ETL process
- Involved in the performance tuning during historical & Daily loads and reduced the batch window and also in making the application Robust.
- Experience in deploying new applications into the production environment.
- Supporting the existing and new DataStage applications in order to smooth functioning of the applications.
- Redesigned few jobs in Datastage Designer to meet the changes in new incoming feeds.
- Involved in importing and exporting jobs category wise and maintaining the backup regularly.
- Involved in unit, system and integration testing of the design.
- Good experience in working as coordinator to offshore team from Onshore under ETL, DataStage, Unix and Oracle environment
Environment: IBM-InfoSphere Datastage 9.1, Oracle 11G, Oracle SQL Developer, DB2, Mainframe files, UNIX- AIX, Secure FTP, Job Control, CA Workstation, ALM.
Confidential, San Antonio, Texas
Datastage Developer
Responsibilities:
- Involved in meetings with the users to gather and analyze the requirements.
- Worked closely with the Data analyst and business analyst during the design and development of ETL technical specification document.
- Used DataStage as an ETL to extract data from sources like flat files and DB2 and loaded to target DB2 UDB.
- Extensively used parallel stages like Aggregator, Sort, Column Generator, Join, Merge, Head and peek for development and de-bugging purposes.
- Developed jobs to standardize and improving error handling.
- Implemented Surrogate key by using Key Management functionality for newly inserted rows in Data warehouse which made data availability more convenient.
- Implement Slowly Changing Dimensions (Type1 and Type2) using DataStage ETL jobs.
- Implemented Surrogate key by using Key Management functionality for newly inserted rows in Data warehouse which made data availability more convenient.
- Developed Jobs to load Staging and ODS tables.
- Responsible for the design, development, coding, testing and debugging of application to meet the requirements of the users.
- Extensive experience in writing complex queries.
- Extensively used SQL tuning techniques to improve the performance.
- Working on improving performances of the jobs to meet strict time lines.
- Involved in unit, system and end-to-end testing of the design.
- Worked with Squirrel to interact with Oracle and used the tool for testing.
- Worked on call for production support.
Environment: Datastage Enterprise 8.5, IBM AIX 5.2, UDB DB2 LUW 9.6, AQT, PL/SQL, UNIX Shell Scripting, Squirrel.
Confidential, NJ
Data Stage Developer
Responsibilities:
- Interacted with Business Analysts to finalize the requirements and documented the technical design document for Coding.
- Translated business processes into DataStage jobs for building data marts.
- Worked with different sources such as Oracle, SQL Server and Excel, Flat, Complex Flat File, COBOL files.
- Designed DataStage ETL jobs for extracting data from heterogeneous source systems, transforming and finally loading into the data marts.
- Identified source systems, their connectivity, related tables and fields and ensured data consistency for mapping.
- Followed the Software Development Life Cycle (SDLC) process in the development of the application. Developed various triggers, functions and stored procedures.
- Created source table definitions in the DataStage repository by studying the data sources.
- Imported and exported repositories across projects.
- Used the Data stage Manager to import and export jobs.
- Written various Triggers, Stored Procedures to perform or automate many of the tasks.
- Tuned DataStage transformations and jobs to enhance their performance.
- Wrote SQL statement and stored procedures in Oracle for extracting as well as loaded data to Oracle tables
- Used Change Data Capture (CDC) technique to capture inserts, deletes and updates and made changes to the target database. Created reusable components like parameter sets.
- Created various Jobs to serve the recurring requests reducing the manual effort and turnaround time.
- Developed UNIX shell scripts to automate the Datastage Jobs to populate/ refresh to the Data warehouse.
- Used DataStage Director and its run-time engine to schedule and execute developed jobs and job sequences, and used log events to monitor job progress and performance.
- Created FDD (Functional design Documents), TDD (Technical Design Document), Issue resolution Trackers and Unit test case document.
Environment: DataStage 7.5.2/8.1, Oracle 9i, 11g, TOAD, SQL/PLSQL, UNIX and Windows NT/2000.
Confidential
ETL Developer
Responsibilities:
- Involved in the design and development of application.
- Created database objects, tables, and indexes with Designer 2000.
- Involved in coding and testing of VIPS.
- Involved in creation and maintenance of database tables.
- Extensively worked with Functions, Packages, and Triggers.
- Also developed PL\SQL procedures to read data from ASCII files and store them in the database.
- Extensively used AWT classes, Event models and exception handling.
- Developed Pro*C programs to generate log file of transferred and changed data.
- Developed Korn shell scripts to automate file manipulation and data loading procedures.
- Involved in the study and understand all the forms and Reports, convert to the GUI, Pro*C.
Environment: Developer 2000(Forms 4.5 and Reports 2.5), Designer 2000, Oracle 7.1, SQL* Loader, PL/SQL, UNIX Window NT.