Sr. Datastage Developer Resume
New, YorK
SUMMARY:
- Over 8 years of experience in System Analysis, Design, Development, Implementation and testing of Databases, Data Warehouse Applications on Client server technologies. Expertise in Software Development Life Cycle (SDLC) and Production Support Life Cycle
- Experience in Data Architecture (involving physical and logical design) of Data Warehouse applications.
- Good hands on experience in RDBMS like Oracle, Netezza, SQL Server, Teradata, and DB2.
- Working experience in data modeling and implementing stored procedures using PL/SQL.
- Extensive knowledge of writing complex queries using SQL.
- Sound knowledge with Netezza SQL with IBM Quality Stage 8.1 and info analyzer 8.1.
- Experience in Data Integration, EDW and Data Mart projects.
- Strong knowledge in OLAP and OLTP Systems, Dimensional modeling using Star Schema and Snowflake Schema.
- Demonstrated work experience in Oracle to DataStage Migration projects.
- Working experience in DB2 Cursors, PL/SQL Triggers and Stored Procedures.
- Expert in working with DataStage Designer and Director.
- Experienced with TOAD and PL/SQL Developer.
- Used Operations Console to monitor the jobs performance and memory / CPU usage.
- Proficient in developing ETL procedures using IBM DataStage 7.5, IBM DataStage 8.x, 9.1 and UNIX Scripts.
- Worked and extracted data from various database sources like Oracle 9i/10g, Flat files, DB2, UDB, XML.
- Experience in writing, testing and implementation of the Stored Procedures, Packages and functions at the Database Level in Oracle 10g.
- Experience in performance tuning both at Application Code Level and Databases Level.
- Experience in designing optimum data marts and data warehouse tables with limited redundancy and for effective and performance efficient reports.
- Have been a team player involved in many enhancements/developments and ensured successful completion of all the projects as per SLA (Service Level Agreements).
- Extensive experience in using Built - in, Plug-in and Custom Stages for Extraction, Transformation and loading of the data.
- Experience in design and implementation of Server and Parallel Jobs including Batches and Job Sequencers.
- Strong communication and analytical skills and a demonstrated ability to handle multiple tasks as well as work independently or in a team towards resolution.
- Extensively Written SQL/PL SQL Queries to extract data from various source systems and populate in SAP reports. Experience in application/web server like Apache Tomcat
- Expertise in implementation of Core Java concept of Java/J2EE
- Good working knowledge in Microsoft SQL Server and UNIX
- Knowledge of XML and related technologies like XSL, XSLT
- Experience working with Log4J
- Sound knowledge of Design patterns like Model View Controller (MVC) architecture
- Experience in SSIS tools like Import and Export wizard, Package Installation and SSIS Package Designer.
- Good knowledge and experience in developing and designing reports through SSRS.
- Efficient in in corporation of various data sources such as Oracle, MS SQL Server and DB2, Teradata, XML, and Flat files into the staging area.
- Experience in Designing, Compiling, Testing, and Scheduling and Running DataStage jobs.
- Expertise in data migration projects with (target - source) same database as well as with heterogeneous databases.
- Experience in Debugging, Troubleshooting and Performance tuning.
- Handled large volumes of data warehouse database
- Ability to adapt to new technologies and environments quickly.
- Good experience in Agile Methodologies, Test Driven Development (TDD).
- Good team player, Strong Interpersonal, Organizational and Communication skills.
TECHNICAL SKILLS:
ETL Tools: IBM DataStage 7.5, 8.0, 8.7 and 9.1 (Designer, Manager, Director, Administrator and Parallel Extender) Web Sphere 8.7/8.5 (DataStage, Information Server, Quality Stage, Information Analyzer, Metadata Server)
Operating Systems: Windows 98/2000,2003, Windows NT 4.0, HP-UX, Linux, AS400 and IBM AIX 4.0, MS Access 2000/2003
Databases: Oracle 10g/9i/8i, DB2 7.2, Teradata V2R6/V2R5, and SQL Server, DB2 V8.1, DB2 V9.0, MS Access.
Languages: XML, XSLT, PL/SQL, SQL, Sun Solaris 5.8/2.7, JAVA
Database Tools: SQL developer, Toad, SQL*Loader, Import, Export
Other Tools: Altova XML Spy, ERwin 7.0/4.1/4.0/3.5 , Toad 8, Autosys 3.0, SQL*Loader, SQL *Plus, IBM Tivoli 8.2, Sterling Commerce, Yantra, Netezza, JIRA, Jenkins
Reporting Tools: COGNOS, SSRS
Version Control: SVN, GIT, TFS.
PROFESSIONAL EXPERIENCE:
Confidential, New York
Sr. DataStage Developer
Responsibilities:
- Designed and tested the DataStage jobs for extract, transform and load.
- Extensively involved in the data migration team to build the Re-usable DataStage job templates, common parameter sets, common DataStage job containers, analyzing with info analyzer 8.1, SQL extract procedures and common re-usable Shell Scripts.
- Worked with business analyst to identify, develop business requirements, transform it into technical requirements and responsible for deliverables.
- Data analysis, Data modeling concepts, Database design and Data migration using Quality stage.
- Expert in using stages, Standardization, Scrubbing, Matching and Survivorship rules using QualityStage.
- Provide the Staging Solutions for Data Validation and Cleansing with PL/SQL and Data Stage ETL jobs.
- Implemented Aggregate, Filter, Join, Lookup, Rank and Update Strategy Transformations.
- Conducted ETL development in the Netezza environment using standard design methodologies.
- Assessed the Netezza environment for implementation of the ETL solutions.
- Used Star Schema and Normalization data modeling techniques for warehousing of data.
- Extensively worked with DataStage 8.7 V - Designer and Director to load data from source extract files to the warehouse.
- Used QualityStage to coordinate the delivery, consistency, removing data anomalies and spelling errors of the source information.
- Created DataStage jobs to load data from sequential files, CSV, Flat files and DB2.
- Cleansed, standardize data using Quality Stage and info analyzer.
- Developed various jobs using Aggregator, Sequential File stages.
- Standardize data to make it easier to find and create one unique view of the business entity with Quality Stage.
- Used FastLoad, MultiLoad Scripts to load data into DB2 from Flat files.
- Worked towards optimal performance when using Stages like LOOKUP, JOIN and MERGE.
- Designed and developed DataStage jobs for Loading Staging Data from different sources like Oracle, DB2 DB into Data Warehouse applying business rules which consists data loads, data cleansing and data massaging.
- Used the DataStage Director and its run-time engine to schedule running the solution, testing and debugging its components and monitoring the resulting executable versions
- Worked with Metadata Definitions, Import and Export of DataStage jobs using DataStage Tools Functionality.
- Created Parameters and Parameter Sets where necessary.
- Tuned DataStage jobs for better performance by creating DataStage Hash files for Staging the data and Lookups.
- Scheduled the server jobs using DataStage Director, which are controlled by DataStage engine and also for monitoring and performance statistics of each stage using IBM Quality Stage 8.1.
- Used Netezza SQL to maintain ETL frameworks and methodologies in use by the company.
- Collaborated with software architects to ensure alignment of the Netezza environment.
- Participated actively in the Team meetings with the developers for bug reviews.
- Developed Shell Scripts to automate file manipulation and data loading procedures.
Environment: IBM Web sphere Data stage 8.7/8.5/8.1/8.0 , Windows NT 2003, Unix SOLARIS, Toad, Oracle 11g/10g/9i, SQL, ERwin, Quality Center, Contorl-M, JCL, SQL Server 2016.
Confidential, Charlotte, NC
DataStage Consultant
Responsibilities:
- Extensively used most of the transforms of DataStage for various types of transformations.
- Validated and successfully executed jobs using DataStage Director.
- Used Quality Stage standardizes stage to reformat data from multiple systems for effective matching and output formatting.
- Worked closely with business users to address their concerns with poorly performing reports. Involved extracting the SQL queries and analyzing their Oracle execution plans and generating Oracle trace files to debug. Develop several Oracle Hint objects to influence queries and alternate methods for report development.
- Set up benchmark reports to understand their pre and post upgrade performance.
- Involved in tuning DataStage repository and jobs for better performance.
- Designed the mappings between sources (external files and databases) to staging targets.
- Used almost all stages like Sequential file, Hash file, OCI9i, ODBC stage etc and also extensively involved in creating custom routines and transforms.
- Constructed SQL Scripts to validate the data after loading process.
- Troubleshooting and support during operational cycle.
- Attended defect logged by the TSG Groups and UAT Users.
- Involved in product support on a rotational basis. Customized UNIX scripts as required for preprocessing steps and to validate input and output data elements, along with DataStage routines. Prepared the test cases for system test.
- Used Quality Stage to parse and standardize the data according to the business requirements.
- Developed Quality stage jobs like investigate, standardize, matching frequency and survive.
- Extracted data from multiple databases like Oracle and Teradata and loaded into Oracle tables.
- Followed Star Schema and Snowflake Schema to develop data marts.
- Extracting, cleansing, transforming, integrating and loading data into data warehouse using DataStage Designer.
- Used DataStage Manager for importing metadata from repository, new job categories and creating new data elements.
- Used DataStage Director and the runtime engine to schedule running the parallel jobs, monitoring and debugging its components for performance and events such as error conditions.
- Involved in the testing of the various jobs developed and maintaining the test log.
- Developed server jobs using server jobs stages like ODBC, Merge, Sort, Hashed file, Transformer, XML input, XML output, XML Transformer, Web Services Client.
- Used Autosys to schedule, run and monitor DataStage jobs.
- Used Clear Case for version control and Clear Quest for bug tracking.
- Worked on troubleshooting, performance tuning and performance monitoring for enhancement of DataStage jobs.
- Designed and developed Servlet components.
- Exposed and consumed SOAP based Web Services to communicate between systems.
- Used Maven for building the application locally and deploying it to local DEV servers.
- Developed various Web Services using Eclipse IDE.
- Worked on reading data from XML files and parsing the XML document.
- Worked on coding, implementation of front end using AJAX, HTML, CSS and JavaScript.
Environment: IBM Info Sphere Information Server DataStage 8.5, 9.1 Quality Stage 8.5, MDM, Oracle 10g/9i, IBM DB2 9.5, Erwin 7.0/4.0, TOAD 9.6, Windows XP, IBM AIX 5.3/5.1, Microsoft Visio, XML, XSLT, SQL, PL/SQL, Unix, SQL Server, Stored Procedures, DB2 UDB, Rational Clear Case and Rational Clear Quest, XML, Java, Maven, Web Services, AJAX, HTML, CSS, JavaScript.
Confidential, Detroit, MI
Data Stage Developer
Responsibilities:
- Expertise in designing and implementing DataStage Architecture in data warehousing and Business Intelligence projects.
- Expertise in IBM Data stage and Quality Stage with data cleansing and transform the data.
- Worked with a Functional Team and Data Modelers/Architects to identify and understand the data from different source systems.
- Process JSON, XML, CSV and Text raw data after cleansing the data.
- Involved in the analysis of the functional side of the project by interacting with functional experts to design and write technical specifications.
- Data base design and base model - Logical & Physical design with hands on experience in DDL and DML SQL operations.
- Used QualityStage stages such as investigate, standardize, match and survive for data quality and data profiling issues during the designing.
- Worked on the Architecture of ETL process with Netezza database.
- Created DataStage jobs (ETL Process) using Info analyzer and Quality Stage for populating the data into the Data Warehouse constantly from different source systems like ODS, Flat files.
- Writes the SQL Queries & PL/SQL Code Tables and Data Mart Staging Tables to validate the data results by counting the number of rows in both the tables.
- Extensively analyzed the Data Sources in identifying data anomalies, patterns, value ranges using Quality Stage.
- UNIX Shell Script to run jobs in multiple instances by using a parameter file.
- Worked on programs for Scheduling Data Loading and Transformations using Data Stage from a legacy system and Data Warehouse to Salesforce Cloud.
- Extracted and load data from Salesforce, Oracle, DB2 databases.
- Preparing development timing plans & reporting to senior management about the supplier progress system & ensuring their engineering support for onsite integration & production launch.
- Worked in a Team with other associate product & component developers.
- Analyzing & Designing, worked on changed requests as per clients and projects technical specification needs.
- Awareness about the functional/business aspects for the components.
- The automated process of job monitoring which helped in minimizing the manual intervention & documenting them perfectly.
- Involved in the Documentation of the ETL phase of the project.
- Extensively Written SQL/PL SQL Queries to extract data from various source systems and populate in SAP reports.
- Developed the reusable components, best practices that were later on used in other Data Warehouse
- Designed and developed complex T-SQL stored procedures for data manipulation.
- Worked extensively on Error Handling and Data Validation.
- Used SQL Server Profiler tools to monitor the performance of SQL Server to analyze the performance of the stored procedures.
- Designed and developed automated Talend Jobs for the automation of various analysis datasets.
- Involved work in UNIX and Windows environment.
- Worked as a lab assistant and assisted students in developing complex queries using SQL including cursors, creating views, synonyms, and indexes.
- Assisted students with creating DDLs and DMLs.
- Evaluated, graded, and maintained student records, successfully completed various academic projects during this tenure.
- Coordinate, lead and assist in the instructional process in preparation and direct interaction with students in lab or problem sessions.
Environment: DataStage 9.1 (Administrator, Designer, Director, Parallel Extender), Data Warehousing, Salesforce, ETL, Oracle 11g, Netezza, Quality Stage 8.1, Info Analyzer 8.1, SQL*Loader, SQL, PL/SQL, Shell Scripts, Windows XP Professional.
DataStage Developer
Confidential
Responsibilities:
- Worked with the Business analysts for requirements gathering, business analysis, testing, and project coordination.
- Involved in creating functional and scope documents for ETL processes.
- Identified and documented data sources and transformation rules required to populate and maintain data warehouse.
- Developed DataStage parallel jobs to load data from sequential files, flat files and Oracle.
- Used DataStage Designer to design and develop jobs for extracting, cleansing, transforming, integrating, and loading data into different Data Marts .
- Used several stages like Sequential file, Hash file, Aggregator, Funnel, Change Capture, Change Apply, Row Generator (debugging stage), Peek, Remove Duplicates, Copy, Lookup and RDBMS during the development process of the DataStage jobs.
- Created parameter sets to group DataStage job parameters and store default values in files to make sequence jobs and shared containers faster and easier to build.
- Performed through data profiling by using the Investigate stage of Quality Stage and also by writing PL/SQL queries to identify and analyze data anomalies, patterns, inconsistencies etc.
- Used multiple invocation ids to run multiple instances of jobs at the same time.
- Designed job sequences to automate the process and document all the job dependencies, predecessor jobs, and frequencies to help the production support people better understand the job runs.
- Migrated jobs from the development instance to testing environment.
- Performed Unit and Integration testing and validated the test cases by comparing the actual results with expected results.
- Used the DataStage Director and its run-time engine to schedule and run the parallel jobs, testing and debugging its components and monitoring the resulting executable versions on an ad hoc or scheduled basis.
- Created Autosys JIL (BOX, CMD, and FWT) Scripts to schedule UNIX Shell scripts.
- Involved in documenting Documents Inspection (DI) results to present to the BA.
- Extensive usage of Toad for analyzing data and writing SQL, PL/SQL scripts performing DDL operations.
Environment : IBM Info sphere Information Server 8.1/8.0(Data Stage, Quality Stage), IBM DB2, UNIX, Windows XP, Teradata, Oracle 9i, and Erwin.
Confidential
Data Stage Developer
Responsibilities:
- Developed Source to Target Mapping documentation based on business requirement specifications.
- Deployed solutions that maximize consistency and re-usability of the Data Stage Jobs.
- Designed ETL jobs to load data into Staging Tables, CODS Core Model, and CODS Consumption Model.
- Developed jobs in IBM Info Sphere Parallel Extender PX v8.1 using different stages like Transformer, Aggregator, Lookup, Join, Merge, Modify, Remove Duplicate, Oracle Enterprise, Sort, Peek, Filter, Copy, Sequential File, Data Set, Surrogate Key Generator etc.
- Designed Data stage sequences to specify Job execution order.
- Designed ETL jobs to identify and remove duplicate records using remove duplicate stage and Generated IDs for the Unique records using Surrogate key Generator Stage.
- Used parallel-shared containers for reusability in other jobs.
- Designed Some Data stage Server Jobs also.
- Extensively used Data stage Director for monitoring and debugging of jobs and sequences.
- Created source to target mapping documents from source to staging area, Staging to CODS Core and Cods Core to Cods Consumption Model mapping Documents.
- Worked on SQL for extracting the data from the Oracle source and Wrote Oracle stored procedures & called the stored procedure in Data Stage.
- Tuned Data stage jobs to obtain better performance.
- Unit Test Data stage Jobs in development including creating the appropriate test data.
- Proactively evaluated the quality and integrity of data required by the users to satisfy their business needs.
- Created and stored the parameters using parameter sets.
- Self-documenting ETL Jobs to comply with Metadata standards.
- Used Autosys Scheduler Tool to schedule Data stage jobs.
- Finally had knowledge transfer meetings to make new team familiar with the project for future Development.
- Worked on Star Schema and Snowflake Schema Models.
Environment: IBM Info Sphere Information Server V8.1 Suite Data Stage and Quality Stage , Autosys, Oracle10g, SQL, PL/SQL, SQL Loader, TOAD, SQL Plus, UIX-HP, Windows-XP.