- Over 8+ Years of experience in IT industry in designing, developing, testing, data quality, enhancement, production support and maintenance of Data Warehouse applications using IBM Infosphere DataStage.
- Extensive experience in ETL IBM InfoSphere DataStage and working knowledge on Data Quality tools including QualityStage and Information Analyzer.
- Good experience in UNIX Korn shell scripting.
- Expert working with cross - functional team for successfully implementation of Data Management and Data Quality projects.
- Good Knowledge about the Architecture and principles of DW like Data marts, OLTP, OLAP, Dimensional Modeling, fact tables, dimension tables and star/snowflake schema modeling.
- Strong knowledge in process like Data Validation, Data Integration, Data Mapping, Data Migration, Profiling and Data Mining.
- Knowledge on Job Sequences to control the execution of the job flow using various Activities & Triggers (Conditional and Unconditional) like Job Activity, Email Notification, Sequencer, Routine activity and Exec Command Activities.
- Excellent in using highly scalable parallel processing infrastructure using parallel jobs with multi-node configuration files.
- Participated in discussions with Project Manager, Business Analysts and Team Members on any technical and/or Business Requirement issues.
- Designed Mapping documents ETL Architecture documents and specifications.
- Experienced in scheduling sequence, parallel and server jobs using DataStage Director, UNIX scripts and scheduling tools.
- Designed and developed parallel job and sequence jobs using DataStage Designer.
- Experience in writing UNIX Shell scripts for various purposes like file validation, automation of ETL process and job scheduling using CA Work Load Automation (Autosys), Automic and Crontab, Control -M and Tidal.
- Extensively made use of Parallel stages like Sequential, Aggregator, Head, Tail, Sort, Lookup, Merge, Join, Change Capture, SCD stage, Peek stages, Dataset, Filter, Enterprise database stages in Parallel Extender job.
- Worked and extracted data from various data sources such as GreenPlum, Sybase IQ, Sybase ASE, Oracle, MS-SQL Server, Teradata, DB2, Legacy systems and Flat files.
- Sound knowledge of Oracle 12c,10g/11g, GreenPlum, SybaseIQ, SybaseASE, DB2 7.0 and good working Knowledge with, Teradata V2R5/V2R6 and MS Sql Server.
- Good knowledge in using PostGreSql, PL/SQL, Teradata SQL to write stored procedures, functions, and triggers and fine tuning.
- Extensive experience in Unit Testing, Functional Testing, System Testing, Integration Testing, Regression Testing, User Acceptance Testing and Performance Testing.
- Created local and shared containers to facilitate ease and reuse of jobs.
- Experience in Technical documentation (Source-Target Mapping, ETL Design, Impact Analysis, Production Support Handover)
- Used the Data Stage Director and its run-time engine to schedule running the solution, testing and debugging its components, and monitoring the resulting executable versions (on an ad hoc or scheduled basis).
- Used IBM InfoSphere Information Analyzer extensively to assess the quality of the data by identifying inconsistencies, redundancies and anomalies in the data at the column, table and cross-table level.
- Expert working on column analysis, rule analysis, primary key analysis, cross-domain analysis and creating data rule definitions, rule set definitions, data rules, rule sets using IBM InfoSphere Information Analyzer
- Extensive experience in loading high volume data and performance tuning.
- Quick learner and adaptive to new and challenging technological environments.
- To take care of the demands & requests raised by Caterpillar business for new ETL component design & development or enhancements/modifications of existing ETL components to support and provide agility to critical business processes through the process of extraction, transformation of data from required sources and loading of data to designated targets of the critical business processes through DataStage. Downstream systems and processes will then utilize the transformed data to drive key business KPIs.
- Create proposal after gathering requirement from stakeholders
- Creating design document and getting it approved after review from client Caterpillar
- Working on development on demands assigned.
- Working with offshore team to get the demand completed when working as solution designer.
- Preparing mapping documents and gathering information for different source and target e.g. SAP connectors, Amazon S3, Hierarchical stage, complex flat file, Snowflake, Teradata connectors.
- Worked on various data warehouses like Equipment (CEDE), Spend (SDW) and Supply chain (SCDW) to cater to the analytical need of different business functions. The program supports middleware application integrations through Data Stage jobs.
- Supporting demand on real-time integrations based on SOAP, REST, XML and FTP protocols and supporting integrations of CAT applications to cloud systems like Salesforce.
Environment : IBM Infosphere DataStage 11.5 (Designer, Director), Teradata, Tufops, WinSCP, Citrix, Microsoft Office, Tidal Scheduler, Teams, VSTS, Postman
- Work with Product Owners on grooming effort for different features, stories and estimate task in Rally (Agile) for development in sprints.
- Coordinate with Offshore team on different task during the sprint.
- Assisting in clarifying issues faced by the team.
- Resolving incidents on PIMS and psearch where business users or members are not finding the right data as expected which include Location, providers, specialties and networks.
- Coordinate with different teams to gather information to accomplish task in the sprint.
- Work on complex sql queries in procedures, packages.
- Worked on migration of Datastage jobs, scripts from 8.5 to 11.3.
- During migration acted as test lead to make sure the data coming as output as same for both versions
- Working on 3NF and Datawarehouse facts tables.
- Working with REST service as some data feed (json format) is coming from the API’s.
- Working with legacy apps e.g Choreo.
Environment : IBM InfoSphere DataStage 11.3 (Designer, Director), Oracle12c, Sybase ASE, GreenPlum, Sql server, Shell Scripts, PUTTY, WinSCP, Citrix, Microsoft Office, GitHub, Skype, Team Foundation Server, Confluence, Postman
Datastage Developer/Data Quality Analyst
- Converted business specification documents to Datastage jobs.
- Worked with Data modelers for design of the tables.
- Worked on deprecated stages to fix the errors and warnings when running jobs.
- Worked on the sql defined in DB2, Sybase ASE (Facets), ODBC and Teradata stages to make them run properly.
- Worked with testers to validate data as per business specifications.
- Formulated and defined system scope and objectives through research and fact-finding to design, develop, modify or integrate complex information systems.
- Designed, coded, tested, debugged and documented programs, subroutines, and scripts.
- Served one or more project team roles for small to medium efforts to manage phases of medium to large efforts in the project.
- Enhanced the design of current systems to improve system capabilities to meet the changing needs of the business.
- Worked with business users or business systems analysts to understand requirements and translate them into technical specifications
- Developed and implemented program/system test plans. Devised data verification methods and standard system procedures.
- Assisted in preparing and maintaining application design document.
- Responded to system failures and performance events by taking appropriate measures to reduce system downtime and eliminate recurrence of problems.
- Conducted presentations to provide end users with knowledge to maximize their use of developed systems.
- Provided technical leadership on assigned projects or discovery efforts.
- Coordinated work assignments and verified results of applications systems analysis and programming personnel.
Environment : IBM InfoSphere DataStage 9.1 (Administrator, Designer, Director), IBM InfoSphere Information Analyzer 9.1, Sybase ASE, DB2, Teradata, Sql server, Teradata sql assistant, Shell Scripts, PUTTY, WinSCP, Citrix, Microsoft Office, Skype, PPM, Rational Clearcase, Sql server Management studio.