Datastage Developer/admin Resume
Dallas, TX
SUMMARY
- ETL Developer with more than 7+ years of experience in the field of ETL design and development. Involved in Gathering of Requirements, System Design, Data Modeling, ETL Design, Development, Production Support and Maintenance.
- Extensive ETL tool experience working with IBM Data Stage 7.5/8.1/8.5/8.7/9.1/11.3/11.5 and worked on Datastage client tools like Datastage Designer, Datastage Director and Datastage Administrator.
- Experienced in scheduling sequence, parallel and server jobs using DataStage Director, UNIX scripts and scheduling tools.
- Good experience in Data Stage Administration, Information Server(IS).
- Experience in Data cleansing by matching user introduced data with database data, removing duplicates, and extraction of relations from source systems using Quality Stage.
- Experience in Data Enrichment and Re - Engineering Using Quality Stage and Datastage.
- Worked with and extracted data from various database sources Oracle, DB2, SQL Server, Teradata, DB2, Mainframe, XML Files, Flat Files, etc.
- Good knowledge in Data warehouse concepts and principle like Star Schema, Snowflake Schema, Slowly Changing Dimension, Surrogate Key, and Normalization etc.
- Experience in ETL design and development of Data Marts and Operational Data Stores.
- Worked extensively with Dimensional modeling, Data migration, Data cleansing, ETL Processes for data warehouses.
- Experience in Troubleshooting of jobs and addressing production issues like data issues, performance tuning and enhancements.
- Used Both Pipeline Parallelism and Partition Parallelism for improving performance.
- Good Experience working with adhoc queries and Optimization.
- Experience on Production Support tools ITIL, Service now, ITSM.
- Good experience on XML in Data Stage.
- Worked extensively on joining tables, multiple data connections using blending, worked on Data Extracts and managing queries.
- Good experience working with Oracle 8i/9i/10g/11g.
- Good SQL/PLSQL programming experience including Performance Tuning, Partitioning, and Index.
- UNIX Shell scripting experience including Archiving, SFTP,DS Job Run.
- Working knowledge of Teradata involving architecture,Teradata SQL, Indexing, Performance Tuning. Familiar with differentTeradata Utilities involving FastLoad, MultiLoad, TPUMP and BTEQ.
- Good Experience on TABLEAU 9.0 with all Drill down and Drop down mechanisms.
- Domain experience in Retail Industry, Insurance and Banking.
- Experience in Enhancements and Migration projects.
- Experience in Requirements Gathering, Business Analysis, Business Process Redesign and Mapping, Testing & User training, Documentation of activities, processes, test scripts and results and Client interaction.
- Knowledge in using PL/SQL to write stored procedures, functions, and triggers.
- Good Working knowledge of SQL, Data Stage, UNIX, Control M.
- Good understanding of Hadoop file system & architecture.
- Knowledge on Data Modeling (Erwin).
- Involved in the development process to ensure that quality is built into the product and verify that the system complies with the functional specification.
- Good knowledge on Creation, Review and Execution of Test Plan, Test Strategy, Test Procedures, Test Cases, Closure Summary.
- Ability to learn new technologies with in a minimal time period and Good team management skills.
- Good communication skills and Effectively interacts with Clients.
TECHNICAL SKILLS
Languages: XML, SQL, PL/SQL, UNIX Shell Script, C, Java.
Database: Oracle 11g/10g/9i, SQL Server 2003/2005/2008 r2, DB2, WINSQL(Netezza), Teradata (SQL Assistant).
ETL: IBM Data stage 11.3/ 9.1,8.7,8.5,8.1,7.5 (Manager, Designer, Director, Administrator).
BI Tools: Tableau 8.0/9.0.
Data Modeling Tools: ERWIN 4.2/4.1/3.x
Scheduling Tools: Control M, Autosys, Crontab.
ITIL Tools: Service now, ITSM.
Web/Apps Servers: IBM Info Sphere Web Sphere.
Data Governance Tools: IBM MDM.
PROFESSIONAL EXPERIENCE
Confidential - Dallas, TX
DataStage Developer/Admin
Responsibilities:
- Provide DataStage Development & Administration activites in Kansas Modular Medicaid System (KMMS) project, the new Medicaid Management Information System (MMIS).
- Developed parallel jobs using most of the stages like Sequential file, change capture, funnel, transformer, data set, ODBC stage etc.
- DevelopedDatastagecode using various stages like Join, Transformer, Sort, Remove Duplicates, Lookup, Aggregator, Funnel, Complex Flat File, modify stage, Peek stage, Remove Duplicate Stage, Sort Stage.
- Designed and Developed Data stage Jobs to Extract data from heterogeneous sources, Applied transform logics to extracted data and Loaded into Data Warehouse Databases.
- Created various UNIX shell scripts to automate the jobs.
- Created Unix Shell Scripts to execute the ds jobs, database connectivity and file management in UNIX.
- Evaluated the Data extraction from source to confirm for data irregularities and classify corrupt data to apply suitable transformation in the mappings in the jobs.
- Understand the business rules completely and implements the data transformation methodology.
- Involved in the Technical design document and Source-To-Target mappings.
- Extensively used Data Stage Designer for Developing Parallel jobs and performed complex mappings based on Business specifications.
- Involved in the design and development of the ETL process for the data warehouse.
- Implemented SCD 1 and SCD 2 Jobs.
- Design and implemented XML Jobs.
- Worked in unit testing, integration testing and user acceptance testing.
- Maintenance of DataStage on theLinux machines Non-Production, Pre-production and Production. And provide support to the daily and weekly production cycles.
- Install, configure and update the IBM InfoSphere Information Server DataStage solution components 11.5. DataStagetechnology support such as troubleshooting product related issues, performance tuning.
- Support and troubleshoot production systems, optimize performance, resolve complex production problems, and provided timely follow-up on problem reports related to DataStage environment.
- Prepare Backup/DR Process. Provided On call support. Outage/Reboot Process.
- Have setup ETL Environment such as project creation, user creation, assigning roles, setup message handler, setting DS environment variables, resource allocation (Scratch and resource disk), set APT configuration and setting purge policy.
- Ensuring best practices are being followed and recommending short-term and long-term solutions for the ETL applicationsPeriodic health checkup and monthly/weekly metrics generation for the DataStage Servers.
- Used the IBM Infosphere DataStageDirector and its run-time engine to schedule running the solution, testing and debugging its components and monitoring the resulting executable versions.
- Unlock DataStage jobs/projects, clearing/killing hanging process. Maintain ownership of release activities which interact with ETL projects.
- Built highly scalable parallel processing infrastructure using parallel jobs with multi-node configuration files.
- Configuration of PX nodes (APT CONFIG FILE) on IBM Information Server 11.5
- Creation of DataStage XMETA Repository on Oracle11g,12c
- Prepared documentation of configuration and procedures as required. Provided Standard documentation, best practices, common ETL Project Templates. Fine Tune jobs/Process to higher performances & debug critical/complex job.
- Support, creation and configuration of IBM DataStage integration tools.
- Involved in Ad-hoc reports as requested by the client by extracting data from Oracle, programming SQL joins and exporting it to excel or text files.
- Developed SQL code and parameterized it to use in the common code approach to avoid writing several SQL scripts for different source table/file formats.
- Configuration of various database connectivity (Oracle11g,12c)
- Set up the projects, roles, users, privileges in different environments (Dev, Test, PROD)
- Maintain & monitor Unix file system for DataStage ETL Projects
- Releasing Lock’s & processes on ETL Job’s
- Create and contact IBM regarding IBM PMR’s
- Debug any issue’s between client and server connectivity’s
- Perform ETL job migrations/Promotions (DSX), Files & Unix Scripts from DEV to QA, QA to PROD environment in compliance with Change Control Board.
Environment: InfoSphere Information Server 11.5, DataStage 11.5, Designer, Director, Unix Shell Scripting, Red Hat Enterprise Linux Server release 6.9, Putty, WinSCP, ServiceNow, Oracle, SQL, SQL Plus, SAP Business Objects XI.
Confidential, Atlanta - GA
ETL DataStage Admin/Developer
Responsibilities:
- The work involved Understanding the Ecommerce BI architecture, preparing LLD and Mapping specifications to design the job.
- The work involved coordinating with different application owners and dc team identifying test scenarios and test flows.
- Developed different Data Stage parallel jobs. By using data from different sources formatted, Cleaned, summarized, aggregated and transform into data warehouse.
- Worked extensively on different types of transformations Change capture, Modify, Stored procedure, Peek, Tail, Head, Sequence generator, Aggregator, Column Import, Column Export, Inter Processer, Quality Stage, and FTP Stage.
- Performance tuning of ETL jobs for optimum performance and Performed unit testing to the job that are developed and provided to support the code when is it in QA.
- Understanding the existing production systems and associated ETL processes.
- Implemented Slowly changing dimensions so as to maintain the history data.
- Implemented various partition methods like Hash, Entire, Auto, db2.
- Built ETL jobs to populate dimension tables (Type1, Type2), FACT tables, and LOOKUP tables.
- Built ETL jobs for migration of data from Source to Target and extensively worked on the Performance issues to enhance the performance.
- Scheduled Job Sequencers to populate data into the dimension tables at specific times.
- It involved an end-to-end testing of inbound and outbound transactions from corporate to Warehouse.
- The work involved gathering requirement from client, formulate testing strategy, test planning, identifying test scenarios, test data set up, ensuring quality deliverables, and giving technical solutions to the team.
- Data validation was done in various systems like WMS, TIPS, Data stage areas.
- Effectively interacts with clients.
- Involved job execution and checking the logs and rectify them.
- Created UNIX scripts to automate ETL jobs and handle the file transfer, and to process the ETL job based on the file availability.
- Monitoring the jobs From Data Stage Director, Control M
- Created folder structure and uploaded all the documents in share point to keep track.
- Status reporting on a daily & weekly basis, defect capture, retesting and closure of defects.
Environment: IBM InfoSphere DataStage 11.3/9.1/8.7 , QualityStage, Oracle 10g/11g, DB2, Erwin 4.1, SQL Server 2008r2/2012, PL/SQL, Windows 2007, XML files, Tableau 8.0, Putty, Sql Assistant (Teradata), Control M.
Confidential, Charlotte - NC
ETL/DataStage Developer
Responsibilities:
- Designed the ETL jobs using IBM Info Sphere Data Stage 8.5 to Extract, Transform and load the data into EDW.
- Design the jobs using different Databases, Processing and File Stages like DB2 connector, Oracle connector, Transformer, Aggregator, Sort and Sequential Files or Data Sets.
- Responsible for data analysis, requirements gathering, report analysis, source-to-target mapping, frequency analysis, process flow diagrams, and documentation.
- Design the jobs using different stages to load the data into Operational Data Stores.
- Design the jobs using different stages like JOIN, LOOK-UP, SORT, AGGREGATE, REMOVE-DUPLICATES and FUNNEL to load the data into EDW.
- Designing the shared container jobs for reusing of business logic.
- Creating the data flows in the design studio by considering various stages and connectors.
- Responsible to create the standard parameter sets with unique standard naming formats.
- Responsible to design the jobs according to the requirements and standards.
- Responsible to create the DSX files of the designed codes.
- Responsible to create the Unit Test plan, conduct the meeting sessions with the testers, and compare the Unit Test results with the different test cases of testers.
- Responsible to create the Deployment plans by including all the pre-production and post-production steps and getting the approval from change management team to deploy the DSX files into QA and Production environment.
- Developed various bulk load and update procedures and processes using SQL * Loader and PL/SQL.
- Created ETL packages with different data sources (SQL Server, Flat Files, Excel source files, XML files etc) and then loaded the data into destination tables by performing different kinds of transformations using SSIS/DTS packages.
- Developed SSIS packages to export data from OLTP database to OLAP database.
- Used SQL Loader to load the data into Oracle tables.
- Developed UNIX shell scripts to automate file manipulation and data loading procedures.
- Responsible for daily verification that all scripts, downloads, and file copies were executed as planned, troubleshooting any steps that failed, and providing both immediate and long-term problem resolution.
Environment: IBM InfoSphere Data Stage 8.5/8.7(Designer, Director), Quality Stage, Oracle 10g/11g, Teradata, SQL*Loader
Confidential
Datastage Developer
Responsibilities:
- Worked with the Business analysts and the DBAs for requirements gathering, analysis, testing, and metrics and project coordination.
- Involved in extracting the data from different data sources like Oracle and flat files.
- Involved in creating and maintaining Sequencer and Batch jobs.
- Creating ETL Job flow design.
- Used ETL to load data into the Oracle warehouse.
- Created various standard/reusable jobs in DataStage using various active and passive stages like Sort, Lookup, Filter, Join, Transformer, aggregator, Change Capture Data, Sequential file, Datasets.
- Involved in development of Job Sequencing using the Sequencer.
- Used Remove Duplicates stage to remove the duplicates in the data.
- Used designer and director to schedules and monitor jobs and to collect the performance statistics.
- Extensively worked with database objects including tables, views, indexes, schemas, PL/SQL packages, stored procedures, functions, and triggers.
- Creating local and shared containers to facilitate ease and reuse of jobs.
- Implemented the underlying logic for Slowly Changing Dimensions.
- Executed Pre-and Post-session commands on Source and Target database using Shell scripting.
- Worked with Developers to troubleshoot and resolve issues in job logic as well as performance.
- Documented ETL test plans, test cases, test scripts, and validations based on design specifications for unit testing, system testing, functional testing, prepared test data for testing, error handling and analysis.
- Understanding existing business model and customer requirements.
- Designed and developed DataStage ETL Parallel Jobs between Source and Target.
- Extensively worked on the Sequential files, to source the data.
- Involved in creating UTP test case document creation.
- Involved in promoting the code from DEV to UAT and to support QA team for every query’s.
- Resolving the defects assigned by the QA and Business team.
- Involved in deploying code to UAT and PROD.
- Involved in UAT & Production support for the earlier releases for Daily loads, pre-verified and Monthly.
Environment: IBM Info Sphere Data Stage 7.5(Administrator, Designer, Director), IBM Information Analyzer, Microsoft SQL 2005, IBM DB2 9.1, Oracle 10g.
Confidential
SQL Developer
Responsibilities:
- Involved in Preparing Technical Design Documents for various interfaces between several applications that have been designed in the project.
- Extensively used Oracle Loader and PL/SQL for ETL.
- Identified business rules for data migration and perform data administration through data models.
- Performance tuning of SQL queries consisting of many tables with large amount of data.
- Created and modified several Database Triggers, especially for Audit and Security Purposes.
- Created Packages, Procedures and Functions that extensively used PL/SQL cursors, user defined object types and exception handling.
- Optimized Query Performance, Session Performance and Reliability.
- Extensive query optimization and performance tuning.
- Resolved several hardcore business rules and issues at the client site.
- Complex SQL queries are used for data retrieval
- Managed database backup and recovery create database objects including tables, views, indexes.
- Experienced in troubleshooting, performance tuning, query tuning, and index tuning using DBCC command, Wizards and SQL Profiler.
- Accomplished maintenance of clustered and non-clustered indexes.
- Developed DTS, SSIS Packages to transform data from excel file to SQL Server database.
- Worked in high availability SQL Server solutions, such as Mirroring, Log shipping, and Replication.
Environment: Oracle, SQL, PL/SQL, Windows 95, UNIX, DB2, and SQL*Plus.