Senior Etl Datastage Developer Resume
Freeport, ME
SUMMARY:
- Highly skilled IT Professional with 7 plus years of experience in analysis, design, development and deployment of large and complex software applications.
- Over 7 years of experience in ETL & Business Intelligence tools and Maintaining of Data Warehouses and Data marts using Data Stage 11.5/9.1/8.1.
- Worked on several Proof Of Concepts to implement Streaming/ETL pipelines/Data Science/Machine Learning and new cutting - edge technologies
- Developed and delivered long-term strategic goals for data architecture vision and standards in conjunction with data users, clients, and other key stakeholders.
- Led ETL technical teams at offshore and onsite by integrating data from several source systems to build Enterprise data warehouse for operational reporting and analytics.
- Expert in all the stages of Software Development Life Cycle (SDLC). Involved in end to end SDLC of project development by providing Gathering Business requirements, Analysis and Design Reviews, Development, Code walkthroughs, Production implementation and Post implementation validation using traditional ETL platforms.
- Experience in design and development of parallel jobs using Parallel Extender for splitting bulk data into subsets and to dynamically distribute to all available nodes to achieve best Job performance.
- Developed efficient mappings for data extraction/transformation/loading (ETL) from different sources to a target data warehouse.
- Experienced in using various stages like Join, Merge, Lookup, Remove Duplicates, Sort, Filter, Funnel, Dataset, Change Data Capture, Switch, Modify, and Aggregator in datastage designing.
- Experience in new enhancements in the IBM WebSphere DataStage - XML, Json, IDOCS, MQ, Web service, Rest API and expert in creating XSD, WSDL.
- Worked on DataStage production job scheduling process using the Scheduling Tool Control M.
- In Depth knowledge in building Enterprise Data Warehouses (EDW), Operational Data Store (ODS), Data Marts, and Decision Support Systems (DSS) addressing Slowly Changing Dimensions (SCDs).
- Excellent knowledge in Extraction, Cleansing and Modification of data from/to various Data Sources like Flat Files, complex files, Sequential files, Comma Delimited files (.csv), XML and Databases like Oracle, SQL Server, Netezza, DB2, Teradata etc
- Extensive experience in writing UNIX shell scripts for data manipulation and transformation.
- Detailed knowledge in Data Warehousing concepts like Dimensional Modeling, Star Schema, Snowflake Schema and ETL.
- Worked with SQL, SQL*PLUS, Oracle PL/SQL Stored Procedures, Triggers, SQL queries and loading data into Data Warehouse/Data Marts.
- Experience in using software configuration management tools like Rational Clear case, SVN, GitHub for version control. Skilled in writing technical specification documents, translating user requirements to technical specification.
- Excellent analytical, interpersonal and communication skills with aptitude to assimilate new technologies.
TECHNICAL SKILLS:
ETL Tools: DataStage 11.5/9.1/8.5/8.1
Databases: Oracle 12c, SQL Server 2008 R2 /2005, UDB DB2, Teradata, IBM Pure Systems Netezza
Reporting Tools: Business Objects, Crystal Reports
Scheduling tool: Control M,ESP
Big Data and Cloud Tool: PIG 0.12.0, Hadoop 2.2.0, Hive 0.12.0, AWS(Amazon Web Services)
Data Modeling Tools: Designer 2000, Star Schema, Snowflake Schema
Version Control: Borland StarTeam, SVN, Git Hub, JIRA. Rational Clear case
Operating Systems: UNIX,LINUX, MS-DOS, Windows NT/XP
Programming Languages: PL/SQL, T-SQL (DDL, DML), XML
PROFESSIONAL EXPERIENCE:
Confidential - Freeport, ME
Senior ETL DataStage Developer
Responsibilities:
- Worked on requirements gathering, analysis, testing, and metrics and project coordination. Developed documents like Source to Target mapping for developing the ETL jobs.
- Populated Data Marts at different levels of granularity for Vendors using DataStage, SQL scripts and stored procedures (being executed from shell scripts driven by Control-M).
- Imported the required Metadata from heterogeneous sources at the project level.
- Involved in the deployment of DataStage jobs from Development to Production environment.
- Developed Parallel jobs using Parallel stages like: Merge, Join, Lookup, Transformer (Parallel), and Oracle Enterprise Stage, Dataset, Funnel, Change Data Capture and Pivot.
- Experienced in using advance DataStage real time stages like SAP IDOC, ABAP, Web services, XML, MQ. Used Regroup, parser, h-join and sort steps in xml.
- Worked with DataStage Designer to create the table definitions for the CSV and flat files, import the table definitions into the repository, import and export the projects, release, and package the jobs.
- Performed debugging on these jobs using Peek stage by outputting the data to Job Log or a stage.
- Used DataStage Designer to create the table definitions for the CSV and flat files, import the table definitions into the repository, import and export the projects, release, and package the jobs.
- Designed and implemented slowly changing dimensions (SCD’s) methodologies
- Wrote complex SQL queries to enable extensive testing of ETL process.
- Developed job sequences to execute a set of jobs with restart ability, check points and implemented proper failure actions.
- Created xsds for file parsing for loading into destination.
- Participated in walk-through and provide approval of Test Plan and Test Cases. Participated in defect reviews.
- Created Control-M UNIX job to schedule the jobs to be run the given time and completion of the dependent jobs.
Environment: DataStage 11.5/9.1/8.7, Oracle 12c/11g, Toad, Squirrel, SQL Server, UNIX, Control-M, Soap UI, MQ Visual Edit, GitHub, Clear case, Jira.
Confidential -Windsor, CT
Datastage Developer
Responsibilities:
- Effectively and efficiently interacted with the client in gathering business requirements for the ETL module.
- Gathering the metadata definition of the source systems, preparing Transformation rules per the business requirements for new enhancements.
- Extracted the data from the Oracle database transforming based on business requirements and loading into downstream Mainframe files/Oracle database for various services to run reports.
- Worked on database connections, SQL joins, views, aggregate conditions, parsing of objects and hierarchies.
- Computed extensive transformation logic within the ETL Jobs using the stages like Sequential file, Dataset, Transformer, Copy, Aggregator, Join, Lookup, Shared Container, XML Input, Funnel, Remove Duplicates and Filter.
- Actively involved in everyday job monitoring and resolved all the issues which are related to job aborts and job failures.
- Used Data Stage Director to validate schedule run and monitor the data stage jobs.
- Created error files & log tables, audit tables containing data with discrepancies to analyze and re-process the data.
- Scheduled jobs in ESP to automate the ETL process
- Involved in the performance tuning during historical & Daily loads, reduced the batch window to make the application Robust.
- Deployed new applications into the production environment. Supported existing and new DataStage applications.
- Redesigned jobs in Datastage Designer to meet the changes in new incoming feeds. Involved in importing and exporting jobs category wise and maintaining the backup regularly.
- Involved in unit, system and integration testing of the design.
- Gained exposure by Working as coordinator to offshore team from Onshore under ETL, DataStage, Unix and Oracle environment
Environment: IBM-InfoSphere Datastage 9.1, Oracle 11G, Oracle SQL Developer, DB2, Mainframe files, UNIX- AIX, Secure FTP, Job Control, CA Workstation, ALM.
Confidential -San Antonio, Texas
Datastage Developer
Responsibilities:
- Involved in meetings with the users to gather and analyze the requirements.
- Worked closely with the Data analyst and business analyst during the design and development of ETL technical specification document.
- Used DataStage as an ETL to extract data from sources like flat files and DB2 and loaded to target DB2 UDB.
- Extensively used parallel stages like Aggregator, Sort, Column Generator, Join, Merge, Head and peek for development and de-bugging purposes. Developed jobs to standardize and improving error handling.
- Implemented Surrogate key by using Key Management functionality for newly inserted rows in Data warehouse which made data availability more convenient.
- Implement Slowly Changing Dimensions (Type1 and Type2) using DataStage ETL jobs.
- Implemented Surrogate key by using Key Management functionality for newly inserted rows in Data warehouse which made data availability more convenient.
- Developed Jobs to load Staging and ODS tables.
- Responsible for the design, development, coding, testing and debugging of application to meet the requirements of the users.
- Extensive experience in writing complex queries.
- Extensively used SQL tuning techniques to improve the performance. Working on improving performances of the jobs to meet strict time lines.
- Involved in unit, system and end-to-end testing of the design. Worked with Squirrel to interact with Oracle and used the tool for testing.
- Worked on call for production support.
Environment: DataStage Enterprise 8.5, IBM AIX 5.2, UDB DB2 LUW 9.6, AQT, PL/SQL, UNIX Shell Scripting, Squirrel.