Lead Datastage Developer Resume
MI
SUMMARY
- Around 11 years of experience in software analysis, requirement gathering, design, development and support of IT applications. Good experience in using various versions of Datastage (11.5/9.1 /8.7 /8.0 /7.5 ).
- Good experience in UNIX Shell scripting, Scheduling with ControlM, TIDEL, SQL Server, Oracle. Experience in handling and execution of Data Integration and Warehouse projects which involve extensive data analysis and Data Cleansing process.
- Coordinated with offshore team for issues resolution, technical reviews and all the offshore deliveries.
- Individually created and submitted ETL proof of concepts (POCs) to the clients before and during execution of projects. Experience in working both on migration and Data Warehousing projects.
- Experience in giving work estimates and creation of documents like requirement specifications, ETL mapping sheets, project description documents, test cases documents, minutes of meetings, review check lists etc. Compile, Surrogate Key Generator Stage, Job Reports, Message Handler Options.
- Experience in doing change control management using tools like PVCS, RTC clients, clear case and other client’s customized tools for the same purpose.
- Practical understanding of the Data modeling (Dimensional &Relational) concepts like Star - Schema Modeling, Snowflake Schema Modeling, Fact and Dimension tables with extensive implementation on Slowly Changing Dimensions - Type I & II in Dimension tables. Good understanding of views, Joins and Partitioning. Knowledge in SQL & PL/SQL and expertise in writing structured queries and handling Netezza database applications. Experience in cross functional systems integration.
- Extensive experience in dealing with High Volume Data, Performance Tuning, Maintaining, Multiple Job. Developed efficient mappings for data extraction/transformation/loading (ETL) from different sources to a Confidential data warehouse.
- Extensively Worked on Job Sequences to control the flow of job execution using various activities like Job Activity, Email Notification, Sequencer, Routine activity and Exec Command. Experience in logical and physical database models. Experience in IBM InfoSphere Datastage 11.5/9.1/8.x/7.X/6.0, Quality Stage (Integrity), Business Objects, Oracle, Netezza, MS SQL Server, MS Access, DB2, SQL, PL/SQL, SQL*Plus.
TECHNICAL SKILLS
Data Warehousing: IBM InfoSphere Datastage 11.5, 9.1/8.x/7.X/6.5, Quality Stage (Integrity), Datamart, OLAP, OLTP, SQL*Plus.
Databases: Oracle 12C/9i/10g/11g, Netezza, Teradata, SQL Server 2000/2005/2008 , DB2, MS Access.
Reporting Tools: SQL, PL/SQL, SQL*Plus, Developer 2000
Scheduling Tool: Control M 9, ControlM 8 & TIDAL
Programming: UNIX Shell Scripting, HTML, XML, SQL, PL/SQL, SQL*Plus, C, Java
Others: HTML, DHTML, XML, ODBC
Environment: Win 3.x/95/98, Win NT 4.0, Win 2003 server, Win 2008 Server, Unix, MS-DOS 6.22.
PROFESSIONAL EXPERIENCE
Confidential, MI
Lead DataStage Developer
Responsibilities:
- Prepared the required application design documents based on functionality required.
- Ensuring team adheres to its chosen process and removes blocking issues.
- To oversee that the team is fully functional and productive; Close cooperation across all roles and functions
- Ensuring that the process is followed, including issuing invitations to daily scrums, sprint reviews, and sprint planning
- Leading and guiding all the team members in all the meetings proposed by agile like preplanning meeting, planning, daily scrum, review and retrospective meetings. So all in all need to be on top of everything.
- Responsible for making sure teams correctly follow the rules and principles that govern scrum.
- Monitoring and resolving any issues that may arise throughout the course of development.
- Tracking the progress through Version one and help the team to achieve the given commitment
- Responsible to deliver maximum value to the customer. Making sure that all involved have the resources customer need.
- Used DataStage Parallel Extender stages namely Sequential, Lookup, Change Capture, Funnel, Transformer Stage, Column Export stage and Row Generator stages in accomplishing the ETL Coding.
- Used Surrogate Keys to keep track of Slowly Changing Dimensions (SCD).
- Developed Teradata SQL Queries to Load data from Teradata Staging to Enterprise Data warehouse
- Designed and developed the jobs for extracting, transforming, integrating, and loading data using DataStage Designer.
- Used the DataStage Director and its run-time engine to monitor the running jobs.
- Involved in performance tuning and optimization of DataStage mappings using features like partitions and data/index cache to manage very large volume of data.
- Involved in Unit testing, System testing to check whether the data is loading into Confidential, which was extracted from different source systems according to the user requirements.
- Extracted the data from the data warehouse using Business Object for reporting purposes.
- Used Extensively Tidal Scheduler to schedule the jobs and to monitor the jobs.
- Developed unit test plans and involved in system testing. Worked on the code fixes and on the tickets raised due to the job failures.
- Supporting unit, integration, and end user testing by resolving identified defects.
- Involved in Implementing ETL standards and Best practices within our portfolio.
- Used the Datastage Designer to develop processes for extracting, cleansing, transforming, integrating, and loading data into data warehouse database. Reusing the logic from Datastage jobs in real time.
- Extensively used almost all database stages, file stages, processing stages, and development/debug stages.
Environment: IBM Web sphere Datastage 11.5 (Designer, Director, Administrator, Manager), Oracle 12C, Windows Server 2008, SQL Developer, TIDAL, SQL, PL/SQL, Oracle SQL Plus, UNIX Shell Scripting, MS SQL Server
Confidential, Eden Prairie, MN
Lead DataStage Developer
Responsibilities:
- Prepared the required application design documents based on functionality required.
- Used DataStage Parallel Extender stages namely Sequential, Lookup, Change Capture, Funnel, Transformer Stage, Column Export stage and Row Generator stages in accomplishing the ETL Coding.
- Developed Teradata SQL Queries to Load data from Teradata Staging to Enterprise Data warehouse
- Extensively worked on Error Handling and Delete Handling.
- Designed and developed the jobs for extracting, transforming, integrating, and loading data using DataStage Designer.
- Used the DataStage Director and its run-time engine to monitor the running jobs.
- Involved in performance tuning and optimization of DataStage mappings using features like partitions and data/index cache to manage very large volume of data.
- Involved in Unit testing, System testing to check whether the data is loading into Confidential, which was extracted from different source systems according to the user requirements.
- Extracted the data from the data warehouse using Business Object for reporting purposes.
- Used Surrogate Keys to keep track of Slowly Changing Dimensions (SCD).
- Extensively used Pivot stage to pivot the source data to achieve required table structures like converting data from rows into a column.
- Used Extensively Tivoli Workload Scheduler to schedule the jobs and to monitor the jobs.
- Developed unit test plans and involved in system testing. worked on the code fixes and on the tickets raised due to the job failures.
- Supporting unit, integration, and end user testing by resolving identified defects.
- Involved in Implementing ETL standards and Best practices within our portfolio.
- Used the Datastage Designer to develop processes for extracting, cleansing, transforming, integrating, and loading data into data warehouse database. Reusing the logic from Datastage jobs in real time.
- Extensively used almost all database stages, file stages, processing stages, and development/debug stages.
Environment: IBM Web sphere Datastage 9.1 (Designer, Director, Administrator, Manager), Terada, Windows Server 2008, SQL Loader, Toad, Control M, SQL, PL/SQL, Oracle SQL Plus, UNIX Shell Scripting, MS SQL Server
Confidential, San Antonio, TX
Lead DataStage Developer
Responsibilities:
- Interacted with business users, reporting analysts in gathering information about the business logic to populate the Confidential tables. Participated in design meetings and perform detail level design.
- Analyzing the requirement and providing estimation based on the complexity.
- Reviewed and updated technical specification documents for ETL process.
- Created source to Confidential mapping and job design documents from staging area to Data Warehouse. Setting up the DataStage project as per the requirement which includes creation of environmental variables, directories, service accounts. Tracking the service requests work status from offshore team.
- Involved in developing DataStage parallel/sequence jobs on 8.1/9.1 version.
- Developed job sequences with restart ability, check points and implemented proper failure actions. Involved in developing UNIX scripts which is required for Control-M scheduling and running ad-hoc jobs. Preparing Unit Testing Plan and executing unit testing cases on the new/modified codes.
- Provided technical support to the operational data source ETL team in solving some of the major issues in DataStage jobs development. Enhancements to the existing jobs as per the service requests.
- Participated in business meetings, raised the requirement gaps and document the process. Split up the task to offshore team and tracing the task through RTC.
- Assigning tasks to the offshore team on development as well as Production Support.
- Uploading the codes into version controlling software (RTC) and getting reviewed by client or peer. Documentation work on migration of codes from Development to Test and Production environment.
- Involved in system testing the codes on the test environment and production validation.
- As a SME, Resolving the escalated failures and providing solution to the Level2.
- Attending change management meeting for code changes every week.
- Preparing job warranty list and providing with back out procedures.
Environment: IBM WebSphere DataStage 9.1,Oracle 10g,Netezza, DB2, SQL, PL/SQL, UNIX Shell Scripting, Datastage Version Control, MS SQL server, Control M 8.0, RTC.
Confidential, San Antonio, TX
Sr. DataStage Developer
Responsibilities:
- Handling production issues and working on developing new design specifications into ETL coding and mapping standards. Implemented incremental extraction and incremental load.
- Extensively used Datastage Designer to develop processes for extracting, transforming, integrating and loading data from various sources into the Data Warehouse.
- Created ETL processes composed of multiple Datastage jobs using job sequencer and developed shell scripts to automate the process and tested the processes.
- Extensively worked on different databases like Oracle, Db2 and SQL Server to extract and load the data from one another. Performed Peer review for Datastage code.
- Used different types of stages like Transformer, CDC, Remove Duplicate, Aggregator, ODBC, Join, Funnel, dataset and Merge for developing different jobs.
- Extensively used Parallel stage like row generator, column generator and Peek Stages for debugging purpose. Used the Director and its run-time engine to schedule running the job, testing and debugging its components, and monitoring the resulting executable versions.
- Documented data sources and transformation rules required to populate and maintain Data Warehouse content. Involved in unit testing, system testing, integration, reporting, and performance testing of the jobs. Replaced transformer stages with other stages to improve performance of job.
- Customized UNIX scripts as required for preprocessing steps and to validate input and output data elements. Developed Multi-Instance reusable Datastage jobs.
- Effectively implemented Partitioning and Parallelism techniques to fully utilize the resources and enhance job performance.
- Wrote SQL scripts to extract and load data from source and Confidential databases.
- Documented Test Plan, Test Cases, Test Scripts and validations based on design specifications for Unit Testing, prepared test data for testing, error handling and analysis.
- Designed and developed various jobs for scheduling and running jobs under job sequencer and Datastage Director. Extensively implemented import/export utilities for migrating code.
- Developed custom routines and transformations for Data Reconciliation of Source.
- Attended daily meetings to review the status of the schedule and go through pending issues. Coordinated with Release manager and DA to migrate the components from one environment to another environment. Implemented slowly change dimensions (SCD) type 1, type 2, type 3.
- Integrated with other Projects, for sharing the table structures and data. Involved in performance tuning by rewriting the queries and modifying existing Datastage jobs.
- Communicated with Offshore team to assign the tasks and get the updates. Worked with String processing in file level using AWK and SED. Facilitate unit, system and user acceptance tests and fix defects by debugging and finding the root cause of the defects in timely manner.
- Involved in production support for production cycle runs and ETL related issues. Communicated data availability with users and management. Documented the purpose of mapping so as to facilitate the personnel to understand the process and in corporate the changes as and when necessary.
- Also involved in ETL test plans, test scripts and validation based on design specifications for unit testing and functional testing. Coordinating with offshore on all activities.
Environment: IBM WebSphere Datastage 9.1/8.1, Oracle 10g,Netezza, DB2, SQL, PL/SQL, UNIX Shell Scripting, Datastage Version Control, MS SQL server, Control M 7.5, RTC.
Confidential
Datastage Developer
Responsibilities:
- Co-ordination between Clients and Offshore team. Documenting the Proof Of Concepts and delivered to Clients. Writing of Technical Specification of the project.
- Design activities for Batch Control Architecture, Mapping Relationships, Keys and Indexes
- ETL Development for Control Architecture, Common Modules, Sequence Controls and major critical interfaces.
- Designed Parallel jobs using various stages like Join, Remove Duplicates, FTP stage, Filter. Extensively used TOAD for analyzing data, writing SQL, PL/SQL scripts performing DDL operations.
- Used Quality Stage for various data cleansing stages to get complete visibility of the actual condition of data, to reformat data from multiple systems to ensure that the data has the correct specified content and format, and to ensure that the best available data survives and is correctly prepared for the Confidential .
- Profiling and minimizing overall costs and resources for critical data integration projects by scanning the samples of data and determining their quality and structure.
- Used development/debug stages to test the environment by creating samples of data from given high volume data or by creating mock data. Ensuring timely deliveries of work items to the Client.
- Worked on the code fixes and on the tickets raised due to the job failures.
- Supporting unit, integration, and end user testing by resolving identified defects.
- Involved in Implementing ETL standards and Best practices within our portfolio.
- Used the Datastage Designer to develop processes for extracting, cleansing, transforming, integrating, and loading data into data warehouse database. Reusing the logic from Datastage jobs in real time.
- Extensively used almost all database stages, file stages, processing stages, and development/debug stages.
Environment: IBM Web sphere Datastage 8.0.1 (Designer, Director, Administrator, Manager), DB2, Oracle, Windows Server 2008, SQL Loader, Toad, Control M, SQL, PL/SQL, Oracle SQL Plus, UNIX Shell Scripting, MS SQL Server
Confidential
Datastage Developer
Responsibilities:
- Understand the technical specifications and developed Datastage Parallel jobs and Quality Stage jobs for Extraction Transformation, Cleansing and Loading process of DW.
- Captured data from a variety of sources including Oracle, Flat Files and Other formats.
- Extensively wrote user-defined SQL coding for overriding for Auto generated SQL query in Datastage.
- Extensively worked with Data Stage Shared Containers for Re-using the Business functionality.
- Analyze the performance of the jobs and project and enhance the performance using standard techniques and Performance Analysis utility. Developed PL/SQL programs and Unix Scripts.
- Used extensively Reject Link, Job Parameters, and Stage Variables in developing jobs.
- Worked with Metadata Management Definitions, Import and Export of Datastage jobs using Data stage Designer. Performed unit testing as part of development using commands wrapped in UNIX scripts.
- Used Various Standard and Custom Routines in Data Stage jobs. Designed jobs to populate the data mart from large files and relational databases.
- Extensively used Scheduling tool Control-M for scheduling Batch jobs to run assigned date and time.
- Developed SQL scripts to supplement the ETL process and to verify on final Confidential data.
- Program Specification Document preparation and review, ETL Development, review and testing, Performance Tuning of the system. Planned and developed Business application using business objects. Developed universes using business objects designer as per business requirements.
- Organized data reports in by using filters, sorting and highlighted data with alerts.
Environment: IIS Datastage 7.5/8.1, IBM DB2 V9.7, OLTP & OLAP process, UNIX Shell Scripting
Confidential
Datastage Developer and Production Support
Responsibilities:
- Technical Solutions provided across projects as and when required within the practice.
- Increasing stability by reducing overall problem counts. Achieve productivity gains, allowing portfolio growth to occur without additional staffing. Fine tuning of ETL Control-M jobs complete handling of new application migration, server patching and system verification activities.
- Deploying new performance monitoring, process improvement jobs in Control M for data warehousing applications. Reporting the status to the Project Manager in weekly basis.
- Monitoring the SLA compliance and make sure that the agreed SLA is being met.
- Planning the process and schedule for the applications turnover to support.
- Updating the clients and respective TTS Managers whenever needed.
- Emailing the status of all the supported Applications to the Application Owners at regular hours. Analyzing the existing production support process, find and fill the gaps and document the same. Making sure that the team is 100% compliant to the quality processes.
- Providing help to Recovery lead in reporting the status to the management.
- Creating and updating documents. Taking turnover of new applications from TTS and transition to the rest of the team. Planning and executing the enhancement activities in the form of SCRs (Service Change Request).
Environment: Datastage 7.5.1, UNIX, DB2, Control M, HPSD, TOAD, PUTTY, SQL developer, Oracle
