Etl Developer/abinitio Developer Resume
Scottsdale, AZ
PROFESSIONAL SUMMARY:
- Over 5+ years of experience in implementing Data warehouse and data base applications with Ab Initio, Informatica, DataStage ETL in addition with data modeling and reporting tools on Teradata, Oracle, DB2, Sybase RDBMS.
- Over 4+ years of experience in full Software Development Life Cycle (SDLC) analysis, design, development, testing, implementation and maintenance in data warehousing environment.
- Highly experienced in ETL tool Ab - Initio using GDE Designer and knowledge of utilizing data warehousing concepts including data modeling and dimensional modeling
- Good experience with Data Migration, Data Transformation and Data Loading using ETL with RDBMS and file systems such as Mainframe, XML, SAP files.
- Very good understanding of Teradata’s MPParchitecture such as Shared Nothing, Nodes, AMPs, BYNET, Partitioning, Primary Indexes etc. Extensively used different features of Teradata such as BTEQ, Fastload, Multiload, SQL Assistant, DDL and DML commands.
- Well versed with Ab Initio parallelism techniques and implemented Ab Initio Graphs using Data parallelism, MFS techniques, Continuous Flows, Component folding and PDL features.
- SQL Database/developer with experience in writing efficient SQL queries, fine tuning queries and wrote several SQL queries for adhoc reporting.
- Expertise in various Ab Initio component groups like Partition, De-partition, Database, Datasets, Transform, FTP, Sort and Miscellaneous.
- Developed various UNIX shell scripts to run Ab Initio and Database jobs. Practical experience with working on multiple environments like production, development and testing.
- Very good experience in Oracle database application development using Oracle 10g/9i/8i/x, SQL, PL/SQL, SQL*Loader. Strong experience in writing SQL, PL/SQL-Stored Procedures, Functions and Triggers.
- Provided Production support and flexible in managing multiple projects.
- Expertise in testing the Ab Initio graphs by using JILs and scheduling in Autosys.
- Configured Ab Initio environment to talk to database using db config, Input Table, Output Table, Updatetable Components.
- Extensively used Ab Initio EME data store/sandbox for version control, code promotion and impact analysis
- Good Knowledge in Dimensional Data modeling, Star/Snowflake schema design, Fact and Dimensional tables, Physical and Logical data modeling.
- Proficient in Teradata TD12.0/TD13.0 database design (conceptual and physical), Query optimization, Performance Tuning.
- Experience in debugging and implementing various techniques in tuning the graphs to improve the performance.
- Experience in using IBM Quality stage (Investigate, Standardize Stage, etc.
- Solid experience in using UNIX and IBM AIX commands and writing UNIX shell scripts.
- Extensive experience working with DataStage Designer and various Stages of DataStage Parallel Extender.
- Extensive use of DataStage client components - DataStage Director, DataStage Manager, DataStageDesigner, DataStage Administrator.
- Experience in handling Mainframe files (EBCDIC).
- Experience in working with various data sources like Sequential file, .ebc file, ODBC, Oracle, SQL Server, DB2, DB2UDB, Teradata and Flat Files in DataStage designer
- Clear and thorough understanding of business processes and workflow.
- Self-initiative, excellent analytical and communication skills, ability to work independently
TECHNICAL SKILLS
Primary Tools: Ab Initio (Co>Op 3.0.3.9/2.15/2.14,Informatica Power Center 9.0.1/8.6/8.1, GDE 3.0.4/1.15/1.14), IBM Information Server 9.1/8.5/8.0.1, Teradata SQL, Teradata Tools and Utilities, Oracle 10g/9i, MS SQL Server 6.5/7.0/2000, Panvalet,Tidal, Appworx
Languages: Teradata SQL, COBOL, JCL, REXX, SQL, BTEQ
Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, SQL Assistant, Viewpoint, Query Monitor
Databases: Teradata 14/13/12/V2R6.2, Oracle 10g/9i, DB2/UDB, SQL Server
Operating Systems: Windows 95/98/NT/2000/XP, UNIX, Linux, NCR MP-RAS UNIX
Data Modeling: Erwin, ER Studio
Scheduling tools: Control M, Autosys
PROFESSIONAL EXPERIENCE
Confidential, Scottsdale, AZ
ETL Developer/Abinitio Developer
Roles & Responsibilities:
- Design and Develop Abinitio graphs to extract data from different sources like Flat files, databases like Teradata, Oracle; Mainframes and XML files, apply transformations and load data into the target systems.
- Efficiently used different Ab Initio components like Reformat, Join, Partition by Key, Partition by Expression, Merge, Gather, Sort, Dedup Sort, Rollup, Scan, FTP, Fuse, Lookups.
- Designing and developing complex sql queries and improvising the performance of existing sql’s by incorporating different optimizing techniques like reducing the joins and using temporary tables and collecting statistics as required.
- Hands-on experience with PDL and metadata programing. Created DML’S, xfrs dynamically as the process is being executed in runtime.
- Create new UNIX scripts to automate and to handle different file processing, editing and execution sequences with shell scripting by using basic Unix commands and ‘awk’, ‘sed’ editing languages.
- Efficiently using the Table extracts using Abinitio and Teradata utilities as required by the process.
- Perform data quality and integrity, end to end testing, performance tuning, debug various production issues and provide support.
- Worked on Abinitio’s Application configuration Environment- ACE, created datasets, lookups and data quality psets which are used as a part of data quality project.
- Good experience in BRE (Business Rules Environment), created rule sets (XFR’s) based on the business requirements that are later used different data quality validations.
- Created a whole new process to run the jobs more frequently, as the files are available and provide the latest updates (for reports) instead of waiting for the next run. In other words providing data for the reports every hour instead of one load per 24hrs.
- Extracted datafrom DB2 database on Mainframes andloadedit into SET and MULTISET tables in the Teradata database by using various Teradata load utilities. Transferred large volumes of data using Teradata FastLoad, MultiLoad, and T-Pump.
- Architected and developed FastLoad and MultiLoad scriptsdeveloped Macros and Stored procedures to extract data, BTEQscripts to take the date range from the database to extract data.
- Created JCL scripts for calling and executing BTEQ, FastExport, Fload, and Mload scripts.
- Developed Teradata BTEQ scripts to implement the business logic and work on exporting data using Teradata FastExport.
- Wrote highly complex SQL to pull data from the Teradata EDW and create AdHoc reports for key business personnel within the organization.
- Created data models for information systems by applying formal data modeling techniques.
- Strong expertise in physical modeling with knowledge to use Primary, Secondary, PPI, and Join Indexes.
- Designed Fact tables and Dimension tables for star schemas and snowflake schemas using ERWIN tool and used them for building reports.
- Loading Data into the Enterprise Data Warehouse using Teradata Utilities such as BTEQ, Fast Load, Multi Load and Fast Export in both mainframes and Unix environments
- Utilized BTEQ for report generation and running the batch jobs as well
- Reviewed the SQL for missing joins & join constraints, data format issues, mis-matched aliases, casting errors.
- Used extensively Teradata Analyst Pack such as Teradata Visual Explain, Teradata Index Wizard and Teradata Statistics Wizard.
- Tuning of Teradata SQL statements using Explain analyzing the data distribution among AMPs and index usage, collect statistics, definition of indexes, revision of correlated sub queries, usage of Hash functions, etc.
- Use SQL to query the databases and do as much crunching as possible in Teradata, using very complicated SQL Query optimization (explains plans, collect statistics, data distribution across AMPS, primary and secondary indexes, locking, etc) to achieve better performance.
- Automated the BTEQ report generation using Unix scheduling tools on Weekly and Monthly basis.
- Developed UNIX shell scripts and used BTEQ, FastLoad, Multiload, and Fast Export utilities extensively to load to target database.
- Created several BTEQ scripts involving derived tables and volatile/Global temporary tables for adhoc purpose to extract data for several business users on scheduled basis.
- With the in-depth expertise in the Teradata cost based query optimizer, identified potential bottlenecks with queries from the aspects of query writing, skewed redistributions, join order, optimizer statistics, physical design considerations (PI/USI/NUSI/JI etc) etc. In-depth knowledge of Teradata Explain and Visual Explain to analyze and improve query performance
- Performed reverse engineering of physical data models from databases and SQL scripts.
- Provided database implementation and database administrative support for custom application development efforts.
- Performance tuning and optimization of database configuration and application SQL by using Explain plans and Statistics collection based on UPI, NUPI, USI, and NUSI.
- Parameterized the graphs to be used in multiple ways and also included conditional components to provide different results for different input parameters with the same graph (Re-usability).
- Optimized Abinitio graphs by looking Confidential the CPU runtime, Skew% available in the logs during the runtime and made necessary changes in the components.
- Monitored AI production jobs like the processing time for different phases and total time and the run flow on WEB-EME. This was very helpful for new code implementations to understand the behavior of new jobs in production environment.
- Monitored Teradata sqls on production using Teradata Viewpoint and provided immediate fixes as required.
- Created Batch Schedule based on requirements from Application/Development Teams.
- Established relationships with all Application/Development Teams to address scheduling requirements using Control-M.
- Detailed documentation of existing, new ETL processes and flows, which are made available for the team members to re-use it in future, scenarios.
- Active team player for many phases of the projects like requirements gathering, Design & Development sessions and also played key role for the Production support.
Environment: Ab Initio (GDE 3.14/3.0/1.14 Co>Op Sys 3.0.4/2.15), Ab Initio - ACE;BRE, Mainframes, DB2, Shell scripts, Unix Korn shell, Teradata, Oracle, SQL Server 2000, XML, AIX UNIX, Solaris, Windows 2000,Control-M
Confidential, Los Angeles, CA
ETL Consultant/Sr.Ab Initio Developer
Roles & Responsibilities:
- Involved in full Software Development Life Cycle (SDLC) - Business Requirements Analysis, preparation of Technical Design documents, Data Analysis, Logical and Physical database design, Coding, Testing, Implementing, and deploying to business users.
- Developed several partition based Ab Initio Graphs for high volume data warehouse.
- Involved in all phases of the System Development Life Cycle Analysis, & Data Modeling.
- Extensively used Enterprise Meta Environment (EME) for version control
- Extensive exposure to Generic graphs for data cleansing, data validation and data transformation.
- Created Sandbox and edited Sandbox parameters according to repository Extensive exposure to EME.
- Used AIR commands to do dependency analysis for all ABI objects
- Involved in Ab Initio Design, Configuration experience in Ab Initio ETL, Data Mapping, Transformation and Loading in complex and high-volume environment and data processing Confidential Terabytes level.
- Developed shell scripts for Archiving, Data Loading procedures and Validation.
- Providing technical support and guidance to the offshore team to address complex business problems.
- Defining the schema, staging tables, and landing zonetables, configuring base objects, foreign-key relationships, complex joins, and building efficient views.
- Expertise in writing scripts for Data Extraction, Transformation and Loading of data from legacy systems to target data warehouse using BTEQ, FastLoad, MultiLoad, and Tpump.
- Developed multiple MapReduce Jobs in . for data cleaning and pre-processing.
- Analyzed large data sets by running Hive queries and Pig scripts
- Performed Query Optimization with the help of explain plans, collect statistics, Primary and Secondary indexes. Used volatiletable and derivedqueries for breaking up complex queries into simpler queries. Streamlined the Teradata scripts and shell scripts migration process on the UNIX box.
- Dealt with initials, delta and Incremental data as well Migration data to load into the Teradata.
- Worked on Informatica Power Center tools - Designer, Repository Manager, Workflow Manager, and Workflow Monitor.
- Using various transformations like Filter, Expression, Sequence Generator, Update Strategy, Joiner, Stored Procedure, and Union to develop robust mappings in the Informatica Designer.
- Developing as well as modifying existing mappings for enhancements of new business requirements mappings to load into staging tables and then to target tables in EDW. Also created mapplets to use them in different mappings.
- Created data models for information systems by applying formal data modeling techniques.
- Strong expertise in physical modeling with knowledge to use Primary, Secondary, PPI, and Join Indexes.
- Designed Fact tables and Dimension tables for star schemas and snowflake schemas using ERWIN tool and used them for building reports.
- Performed reverse engineering of physical data models from databases and SQL scripts.
- Working on different tasks in Workflows like sessions, events raise, event wait, e-mail, command, workletsand scheduling of the workflow.
- Creating sessions, configuring workflows to extract data from various sources, transforming data, and loading into enterprise data warehouse.
Environment:Ab Initio GDE (3.0.4/ 1.15/1.14/1.13), Co>Op (3.0.4.2/ 2.15/2.14/2.13), EME, DB2,Teradata 14.0 (FastLoad, MultiLoad, FastExport, BTEQ), Teradata SQL Assistant, Hadoop-Hdfs, Apache Pig, Sqoop, Flume, Hive, Map Reduce, Informatica Power Center 9, Unix, SQL, PL/SQL, Work Load Manager, MS Access, UNIX.
Confidential, MD
Teradata/Informatica Developer
Roles & Responsibilities:
- Development of scripts for loading the data into the base tables in EDW and to load the data from source to staging and staging area to target tables using FastLoad, MultiLoad and BTEQ utilities of Teradata. Writing scripts for data cleansing, data validation, data transformation for the data coming from different source systems.
- Performed application level DBA activities creating tables, indexes, monitored and tuned TeradataBETQ scripts using Teradata Visual Explain utility.
- Written complex SQLs using joins, sub queries and correlated sub queries. Expertise in SQL Queries for cross verification of data.
- Developed the Teradata Macros, Stored Procedures to load data into Incremental/Staging tables and then move data from staging into Base tables
- Performed Space Management for Perm & Spool Space.
- Reviewed the SQL for missing joins & join constraints, data format issues, mis-matched aliases, casting errors.
- Developed procedures to populate the customer data warehouse with transaction data, cycle and monthly summary data, and historical data.
- Dealt with initials, delta and Incremental data as well Migration data to load into the Teradata.
- Analyzing data and implementing the multi-value compression for optimal usage of space.
- Query Analysis using Explain for unnecessary product joins, confidence factor, join type, order in which the tables are joined.
- Very good understanding of Database Skew, PPI, Join Methods and Join Strategies, Join Indexes including sparse, aggregate and hash.
- Used extensively Teradata Analyst Pack such as Teradata Visual Explain, Teradata Index Wizard and Teradata Statistics Wizard.
- Used extensively Derived Tables, Volatile Table and GTT tables in many of the ETL scripts.
- Tuning of Teradata SQL statements using Explain analyzing the data distribution among AMPs and index usage, collect statistics, definition of indexes, revision of correlated sub queries, usage of Hash functions, etc…
- Flat files are loaded into databases using FastLoad and then used in the queries to do joins.
- Use SQL to query the databases and do as much crunching as possible in Teradata, using very complicatedSQL Query optimization (explains plans, collect statistics, data distribution across AMPS, primary and secondary indexes, locking, etc) to achieve better performance
- Excellent experience in performance tuning and query optimization of the Teradata SQLs.
- Worked on InformaticaPower Center tools - Designer, Repository Manager, Workflow Manager, and Workflow Monitor.
- Using various transformations like Filter, Expression, Sequence Generator, Update Strategy, Joiner, Stored Procedure, and Union to develop robust mappings in the Informatica Designer.
- Developing as well as modifying existing mappings for enhancements of new business requirements mappings to load into staging tables and then to target tables in EDW. Also created mapplets to use them in different mappings.
- Working on different tasks in Workflows like sessions, events raise, event wait, e-mail, command, worklets and scheduling of the workflow.
- Creating sessions, configuring workflows to extract data from various sources, transforming data, and loading into enterprise data warehouse.
- Running and Monitoring daily scheduled jobs by using Work Load manager for supporting EDW(Enterprise Data Warehouse) loads for History as well as incremental data.
- Design, Development and Documentation of the ETL (Extract, Transformation & Load) strategy to populate the Data Warehouse from the various source systems.
- Prepared data marts on policy data, policy coverage, claims data, client data and risk codes.
- Extensively used Informatica PowerCenter 8.6 to create and manipulate source definitions, target definitions, mappings, mapplets, transformations, re-usable transformations, etc.
- Based on the logic, used various transformation like Source Qualifier, Normalizer, Expression, Filter, Router, Update strategy, Sorter, Lookup, Aggregator, Joiner, XML, Stored procedure transformations in the mapping.
- Involved in performance tuning of mappings, transformations and (workflow) sessions to optimize session performance.
- Developed Informatica SCD type-I, Type-II and Type III mappings and tuned them for better performance. Extensively used almost all of the transformations of Informatica including complex lookups, Stored Procedures, Update Strategy, mapplets and others.
- Supported the code after postproduction deployment.
Environment:Informatica 8.5.1, Oracle 10g, Informatica Power Center 8.1.1, Teradata, Erwin 4.5, Oracle Applications, Flat files, PL/SQL, TOAD 9.0, SQL,UNIX, Mainframes (JCL Jobs), Quality Center 9.0
Confidential, Stamford, CT
Informatica/Teradata Developer
Roles & Responsibilities:
- Used various transformations like Filter, Expression, Sequence Generator, Update Strategy, Joiner, and SQL, Lookup (File and Database) to develop robust mappings in the Informatica Designer.
- Worked on Teradata and its utilities - tpump, fastload through Informatica. Also created complex Teradata Macros
- Used various transformations like Filter, Expression, Sequence Generator, Update Strategy, Joiner, and SQL, Lookup (File and Database) to develop robust mappings in the Informatica Designer.
- Worked and Implemented Pushdown Optimization (PDO) to optimize performance issues of complex mappings involving numerous transformations and hence degrading the performance of the session.
- Involved in Performance tuning Confidential source, target, mappings, sessions, and system levels.
- Exhaustive testing of developed components.
- Worked on the various enhancements activities, involved in process improvement.
- Used Informatica client tools - Source Analyzer, Warehouse designer, Mapping designer, Transformation Developer, WorkFlow Manager, Workflow Monitor.
- Working with Power Centre Versioning (Check-in, Check-out), Querying to retrieve specific objects, maintaining the history of objects.
- Worked on Ab Initio in order to replicate the existing code to Informatica.
- Implemented Teradata MERGE statements in order to update huge tables thereby improving the performance of the application.
- Applied different types of monitoring for Power exchange processes like dtlcacon, dtllst, oracle scn, lag scripts, Heartbeat table to monitor the lag
- Created and managed different Power exchange directories like condense files directory, check point directory etc.
- Developed wrapper shell scripts for calling Informatica workflows using PMCMD command and Created shell scripts to fine tune the ETL flow of the Informatica workflows.
- Involved in building tables, views and Indexes.
- Involved in ad hoc querying, quick deployment, and rapid customization, making it even easier for users to make business decisions.
Environment::Informatica 9.1(Designer, Repository Manager, Workflow Manager, Workflow Monitor), Teradata, UNIX, Citrix, Toad, Putty, PL/SQL Developer