Sr. Etl/talend Developer Resume
Irvine, CA
SUMMARY
- Over 10 Plus years of IT experience in Data warehousing with emphasis on Business Requirements Analysis, Application Design, Development, testing, implementation and maintenance of client/server Data Warehouse and Data Mart systems
- Experience in various stages of System Development Life Cycle (SDLC) and its approaches like Waterfall, Agile Model.
- Experience in development and design of ETL (Extract, Transform and Loading data) methodology for supporting data transformations and processing, in a corporate wide ETL Solution using Informatica PowerCenter 9.6.1/9.5.1/9.1/8. 6/7.1 , PowerExchange 9.1, Talend 6.4/6.2, Datastage 11.3
- Experience in OLTP Modeling (2NF,3NF) andOLAP Dimensional modeling (Star and Snow Flake) using ERwin Standard Edition/r7.3/4/3.5 (conceptual, logical and physical data models)
- Experience in integration of various data sources definitions like SQL Server, Oracle, MYSQL, Flat Files, XML and XSDs, Oracle EBS
- Extensively worked on developing InformaticaMappings, Mapplets, Sessions, Worklets and Workflows for data loads
- Created Mappings in Mapping Designer to load data from various sources using complex transformations like transaction control, Lookup (Connected and Un - connected), Joiner, sorter, Aggregator, Update Strategy, Filter and Router transformations
- Expertise in Installing and Managing Informatica Power center, Metadata Manager, Data Explorer and Data Quality
- Experince in Big Data technologies like Hive,Sqoop .
- Experience with relational databases such as Oracle 8i/9i/10g/11g, SQL SERVER 2005/2008,MY SQL, Greenplum and DB2
- Implemented Slowly Changing dimension types I, II III methodologies for accessing the full history of accounts and transaction information
- Strong skills in SQL, PL/SQL packages, functions, stored procedures, triggers and materialized views to implement business logic in oracle database.
- Experience with Database SQL tuning and query optimization tools like Explain Plan
- Experience in SQL*Loader, UTL File Concepts, Import and SQL*Plus, DBMS Packages
- Experience in moving the Program Files (FTP) to the server by using the Tools like WINSCP, Putty and Telnet
- Experience in evaluating data profiling, cleansing, integration and extraction tools(Informatica, Kalido, and Composite Software)
- Involved in writing various UNIX shell scripts for writing automated scripts for scheduled queue process and pmcmd commands
- Experience in debugging and performance tuning of sources, targets, mappings and sessions.
- Experience in identifying and resolve ETL production root cause issues. Experience in maintenance, enhancements, performance tuning of ETL code
- Experienced In working with various Scheduling tools like Autosys, Control-M,Informatica Scheduler
- Developed test case’s for business and user requirements to perform System/ Integration / Performancetesting
- Continuously monitor the accuracy of the data and the content of the delivered reports.
- Excellent communication and interpersonal skills. Ability to work effectively while working as a team member as well as individually.
TECHNICAL SKILLS
Operating Systems: Windows(NT,2000/03/XP/Vista/7), Mac (10.4/10.5/10.6 ), Linux(Red Hat), UNIX (Solaris, AIXv5.2,SunOS 5.10,)
Languages: SQL, PL/SQL, T-SQL, UNIX Shell Scripts, Perl Scripting,Java, XML, Sqoop, Hive
AWS: S3, Lamda, StepFunctions
ETL Tools: Informatica PowerCenter 9.6.1/9.5.1/9.1/8. x/7.1, Talend Data Mangement 6.4, Talend Big data 6.2, Datastage 11.3
Data Modeling: Dimensional Data Modeling (Star Schema, Snow-Flake, FACT, Dimensions), Physical and Logical Data Modeling, Entities, Attributes, Cardinality, ER Diagrams,ERWin StandardEdition/r7.3/4/3.5, MS-Visio 2010/2007
Databases: Oracle 11g/10g/9i/8i, Siebel, MS SQL Server 2005/2008/2008 R2, DB2, MYSQL, Greenplum, AWS Redshfit, Netezza
Scheduling Tools: Autosys, Control-M, Informatica Scheduler
Reporting Tool: Business Objects, Tableau, OBIEE
Others: Web Services, MS Office, MS Visio, TOAD, FTP, SFTP, SCP,GIS, MKS, ALM,TortoiseSVN 1.7.9,PAC2000 v7.6., GIT Lab
PROFESSIONAL EXPERIENCE
Confidential, Irvine, CA
Sr. ETL/Talend Developer
Responsibilities:
- Involved in End-End development of the implementation and Roll out.
- Worked with Data mapping team to understand the source to target mapping rules.
- Analyzed the requirements and framed the business logic and implemented it using Talend.
- Involved in ETL design and documentation.
- Created Talend Development Standards. This document describes the general guidelines for Talend developers, the naming conventions to be used in the Transformations and also development and production environment structures.
- Analyzed and performed data integration using Talend open integration suite.
- Worked on the design, development and testing of Talend mappings.
- Created ETL job infrastructure using Talend Open Studio.
- Worked on Talend ETL and used features such as Context variables & Talend components like tReplace, tmap, tsort and tFilterColumn, tFilterRow,tJava,tjavarow, tConvertType, tjava, txmlMap, tdelimited etc.
- Used Database components like tRedshiftRow, tRedhsiftBulkExec,tRedshiftOutputBulk, tRedshiftClose, tRedshiftConnection, tRedshiftUnlaod, tRedshidtInput, tRedshiftOutput, tRedshiftCommit etc.
- Worked with AWS S3 components like ts3Connection, ts3Get, ts3Put,ts3List.
- Worked with various File components like tFileCopy, tFileCompare, tFileExist, tFileDelete,tFileRename, tFileInputXML.
- Worked on improving the performance of Talend jobs.
- Created triggers for a Talend job to run automatically on server.
- Worked on Exporting and Importing of Talend jobs using Talend Admin Console.
- Created jobs to pass parameters from child job to parent job.
- Monitored the daily runs, weekly runs and adhoc runs to load data into the target systems .
- Created Talend jobs using the dynamic schema feature.
- Created Talend jobs to copy the files from one server to another and utilized Talend FTP components.
- Worked on Joblets (reusable code) & Java routines in Talend
- Implemented Error handling in Talend to validate the data Integrity and data completeness for the data from the Flat File.
- Used Talend Admin Console Job conductor to schedule ETL Jobs on daily, weekly, monthly and yearly basis (Cron Trigger)
- Involved in end-to-end testing of jobs.
- Wrote complex SQL queries to take data from various sources and integrated it with Talend.
- Scheduled the Talend jobs using AWS Cloud watch..
- On-call support for production maintenance.
Environment: Talend Data Management Platform 6.4,AWS S3, Datastage 11.3, Netezza, AWS Redshift, SQL Server 2008, Unix, XML, Json, Falt files, AWS Lamda, AWS Step functions, Tableau.
Confidential, Santa Monica, CA
Sr.ETL/Talend Developer
Responsibilities:
- Participated in User meetings, gathering requirements, discussing the issues to be resolved.
- Participated in the walkthroughs of the data models, ETL specifications, identification of dimensional attributes Facts and Star Schema to fulfill requirements.
- Preparation of Functional Design documents and provides them to offshore for development.
- Participate in Design Reviews of Data model and Informatica mapping design.
- Assisted in creating Logical and Physical data modeling.
- Have been involved in designing & creating hive tables to upload data in Hadoop and process like merging, sorting and creating, joining tables
- Migrated 14 Million Customer & Coaches and 1 .5 Million Orders data from Bydesign, Liferay, WordPress and Infusion Soft to Oracle EBS.
- Performed data manipulations using variousTalendcomponents like tMap, tJavaRow, tjava, tOracleRow, tOracleInput, tOracleOutput, tMSSQLInput and many more.
- Have used AWS components (Amazon Web Services) - Downloading and uploading data files (with ETL) to AWS system using S3 talend components.
- Worked with various File components like tFileCopy, tFileCompare, tFileExist, tFileDelete,tFileRename.
- Designed and customized data models for Data warehouse supporting data from multiple sources on real time
- Creation and maintenance of Informatica users and privileges.
- Created Groups, roles, privileges and assigned them to each user group.
- Migrated Facilities, Classes, Master Trainer, Event, Product data from existing MYSQL to AWS Platform.
- Used Sqoop to export data into HDFS and Hive.
- Developed mappings to load into staging tables and then to Dimensions and Facts.
- Create hive scripts to process various types of data sets and load them into data warehouse built on Hive.
- Designed and created new Informatica jobs to implement new business logic into the existing process
- Using Informatica modules (Repository Manager, Designer, Workflow Manager and Workflow Monitor) to accomplish end to end ETL process.
- Performed data profiling with Sources to analyse the content, quality and structure of source data
- During mapping development.
- Created required scripts/transformations to extract the source data from various sources such as Oracle, Flat Files etc.
- Used Informatica features to implement Type 1, 2 changes in slowly changing dimension, Change Data Capture (CDC)
- Designed and developed complex aggregate, joiner, lookup, Update Strategy, Router, Joiner, Java, SQL transformations to implement the business rules.
- Defined Target Load Order Plan for loading data into Target Tables
- Used Mapplet and Reusable Transformations prevent redundancy of transformation usage and maintainability.
- Used Parameter files to reuse the mapping with different criteria to decrease the maintenance
- Created Connected, Unconnected and Dynamic lookup transformation for better performance and increased the cache file size based on the size of the lookup data.
- Created Complex Informatica mappings and in other hand Simple mappings with Complex SQLs in it based on the need or requirement of business user.
- Used Debugger to test the mappings and fixed the bugs.
- Examining the session logs if the workflows fail and fixing up the defect.
- Worked on transformations like Lookup, Sequence Generator, Filter, Expression, Source Qualifier, Router Update Strategy etc.
- Identified process bottlenecks and implemented performance tuning at mapping and session levels
- Performance tuning of Informatica mappings for large data files by managing the block sizes, data cache sizes, sequence buffer lengths and commit interval.
- Wrote UNIX shell Scripts & PMCMD commands for FTP of files from remote server and backup of repository and folder
- Created UNIX shell scripts to trigger the workflows, parse the files and monitor the loads.
- Created various batch Scripts for scheduling various data cleansing scripts and loading process.
- Expertise in doingUnit Testing,Integration Testing,System TestingandData Validationfor Developed Informatica Mappings.
- Provided production support by monitoring the processes running daily.
- Scheduled the entire Job process with Autosys Scheduler.
- Involved in Informatica Mappings migration from DEV to QA, QA to UAT & UAT to Prod.
Environment: Informatica Power Center 9.6.1, Talend Big Data Studio 6.2, Oracle 11g, SQL,PL/SQL, TOAD, MY SQL, Unix, OBIEE, Oracle EBS, Hadoop, Hive, Autosys Scheduler, AWS Redshift
Confidential, Los Angeles, CA
Sr. ETL Developer
Responsibilities:
- Involved in all phases of software development life cycle.
- Working with business analysts to gather business requirements and implementing the same into a functional data warehouse design.
- Involved in creating Logical and Physical design of data models using Visio.
- Interacting with the clients on a regular basis to discuss day-to-day issues and matters
- Involved in data analysis and handling the ad-hoc requests by interacting with business analysts, clients and customers and resolve the issues as part of production support
- Conduct training and knowledge sharing sessions for the onsite an offshore developers and testers on financial domain and functional areas.
- Actively involved in requirements gathering and complete life cycle of the project.
- Involved in Migration of Informatica 8.6.1 to Informatica 9.5.
- Extensively worked on Informatica client tools - Source Analyzer, Warehouse Designer, and Transformation developer, Mapplet Designer, Mapping Designer, Workflow Designer, Worklet Designer and Task Developer.
- Developed new and maintaining existing Informatica mappings and workflows based on specifications.
- Integrated the data from relation databases (ORACLE, SQL Server, Mainframe, Flat files) for populating large scale DataMart and Data warehouse.
- Created Mapplets, reusable transformations, Worklets and used them in different mappings, workflows.
- Used Source qualifier, Expression, Aggregator, Lookup, Router, Normalizer, Sorter, Stored Procedure transformations to do necessary data calculations and manipulations according to the business rules and loaded data into Target systems.
- Worked with various look up cache like Static Cache, Persistent Cache, Re Cache from database and Shared Cache.
- Successfully Loaded Data into different targets from various source systems like Oracle Database, Flat files, XML files...etc. into the Staging table and then to the target database
- Performed truncate & Load process by using stored procedure transformation and load control table to stage the data.
- Created different parameter files and changed Session parameters, mapping parameters, and variables at run time.
- Worked on Slowly Changing Dimensions i.e. Type1 & Type2.
- Performed incremental aggregation to load incremental data into aggregate tables.
- Developed PL/SQL scripts, stored procedures, Indexes, Constraints, partitions and triggers in Oracle.
- Written PL/SQL procedures in Greenplum as per the requirement.
- Involved in the creation of Greenplum functions and views for the customer use.
- Created DDL, DML for Greenplum database.
- Worked on production issues like bug fixing, bottlenecks, data validation and report errors.
- Performance tuning of Sources, Targets, Mappings and Sessions by identifying bottlenecks and used Debugger to debug the complex mappings and fix them.
- Created pass through session partitions to improve performance of reading source data.
- Extensively used Shell scripts to automate the Pre-Session and Post-Sessions processes.
- Prepared test Scenarios and Test cases in HP Quality Center and involved in unit testing of mappings, system testing and user acceptance testing.
- Scheduled the entire process in Autosys.
- Worked with reporting team to help understand them the user requirements on the reports and the measures on them.
- Managed the 3 team off shore support group in India.
- Involved in designing and developing the reporting requirements by using Business Object reports.
- Worked as a part of a team and provided 7 x 24 production support.
Environment: Informatica Power Center 9.5/8.6.1, Erwin, MS Visio, Oracle 11g/10g, SQL,PL/SQL, TOAD, MY SQL, GREENPLUM, SQL Server 2008, Mainframe, SAP, XML, Business Objects XIR3,Autosys, UNIX Shell Scripting, MKS Integrity.
Confidential, Santa Monica, CA
ETL Lead
Responsibilities:
- As a consultant studied the existing DataMarts to understand and integrate the new source of data.
- Managing the off shore support group in India for support issue as well as small enhancements for data warehouse.
- Preparing the weekly status report and coordinating weekly status calls with technology lead/business
- Designed and created new Informatica jobs to implement new business logic into the existing process
- Using Informatica modules (Repository Manager, Designer, Workflow Manager and Workflow
- Monitor) to accomplish end to end ETL process.
- Performed data profiling with Sources to analyse the content, quality and structure of source data
- During mapping development.
- Created required scripts/transformations to extract the source data from various sources such as Oracle, Flat Files etc.
- Used all the complex functionality of informatica (Mapplets, Stored Procedures, Normalizer,
- Update Strategy, Router, Joiner, Java, SQL Transformation Etc...) to interpret the business logic into
- The ETL mappings.
- Designed and developed complex aggregate, joiner, lookup transformations to implement thebusiness rules in the ETL mappings to load the target Facts and Dimensions.
- Defined Target Load Order Plan for loading data into Target Tables
- Used Mapplets and Reusable Transformations prevent redundancy of transformation usage and maintainability.
- Created Complex Informatica mappings and in other hand Simple mappings with Complex SQLs in it based on the need or requirement of business user.
- Used Informatica’s features to implement Type 1, 2 changes in slowly changing dimension, Change Data Capture (CDC)
- Fine-tuned the session performance using Session partitioning for long running sessions.
- Implemented performance tuning logic on Targets, Sources, mappings, sessions to provide maximum efficiency and performance.
- Used Versioning, Labels and Deployment group in the production move process.
- Automation of Workflow using UNIX scripts using PMCMD, PMserver commands.
- Setup Permissions for Groups and Users in all Environments (Dev, UAT and Prod).
- Createdtables, views, primary keys, indexes, constraints, sequences, grants and synonym.
- Involved in developing optimized code using PL/SQL for Server related Packages to centralize the application through procedures containing PL/SQL were created and stored in the database and fired off when contents of database were changed.
- Used debugger to test the mapping and fixed the bugs.
- Conducted Design and Code reviews and extensive documentation of standards, best practices and
- ETL Procedures.
- Used Workflow Manager for Creating, Validating, Testing and running the sequential and concurrent Sessions and scheduling them to run at specified time.
- Developed Oracle Stored Procedures, Packages and Functions and utilized in ETL Process.
- Handled the performance tuning of Informatica Mappings at various level to accomplish theestablished standard throughput.
- Analysed the Target Data mart for accuracy of data for the pre-defined reporting needs.
- Wrote complex SQLs to achieve and interpret the reporting needs into the ETL Process. Also worked n SQL tuning to achieve the maximum throughput.
- Assisted in all aspects of the project tomeet the scheduled delivery time..
- Provided Knowledge Transfer to the end users and created extensive documentation on the design, development, implementation, daily loads and process flow of the mappings.
- Conducted unit testing of all ETL mappings as well as helped QA team in conducting their testing.
- Wrote UNIX shell scripts to work with flat files, to define parameter files and to create pre and post session commands.
- Used Autosys Tool to schedule shell scripts and Informatica jobs.
- Performed Unit, Grid Integration, Testing and validate results with end users.
- Worked as a part of a team and provided 7 x 24 production support.
Environment: Informatica Power Center 9.5, Erwin, MS Visio, Oracle 11g, SQL, PL/SQL, Oracle Sql Developer Tool, SQL Server 2008, Flat Files, XML, Autosys, UNIX Shell Scripting, Subversion.
Confidential
ETL/Oracle Developer
Responsibilities:
- Involved in all phases of software development life cycle.
- Involved in requirements gathering and analysis to define functional specifications.
- Created various logical and physical data models interacting with the business team, leads and developers using ERWIN.
- Created technical specs documents for all the mappings using MS Visio 2010.
- Involved in extracting source data from Oracle, SQL Server 2000, DB2,Flat Files, VSAM files on different systems
- Developed mappings, sessions, workflows and workflow tasks based on the user requirement and scheduled the workflows in workflow manager and monitor the load status in the workflow monitor.
- Defined Target Load Order Plan for loading data into Target Tables.
- Worked with various look up cache like Dynamic Cache, Static Cache, Persistent Cache, Re Cache from database and Shared Cache.
- Used Informatica’s features to implement Type 1, 2changes in slowly changing dimension tables.
- Created and configured workflows, worklets & Sessions to transport the data to target systems using Informatica Workflow Manager.
- Used Mapplets and Reusable Transformations prevent redundancy of transformation usage and maintainability.
- Fine-tuned the session performance using Session partitioning for long running sessions.
- Implemented performance tuning logic on Targets, Sources, mappings, sessions to provide maximum efficiency and performance.
- Created tables, views, primary keys, indexes, constraints, sequences, grants and synonym.
- Involved in developing optimized code using PL/SQL for Server related Packages to centralize the application through procedures, functions and packages.
- Created Oracle Stored Procedure to implement complex business logic for good performance and called from Informatica using Stored Procedure transformation
- Created Connected, Unconnected and Dynamic lookup transformation for better performance and increased the cache file size based on the size of the lookup data.
- Done various optimization techniques in Aggregator, Lookup, and Joiner transformation
- Implemented Informatica parameter files to filter the daily data from the source system.
- Developed Workflow Tasks like reusable Email, Event wait, Timer, Command, and Decision.
- Used Informatica debugging techniques to debug the mappings and used session log files and bad files to trace errors occurred while loading.
- Used pmcmd command in UNIX scripts to call Informatica session and workflows.
- Creating Test cases for Unit Test, System Integration Test and UAT to check the data.
- Used various Oracle Index techniques like B*tree, bitmap index to improve the query performance and created scripts to update the table statistics for better explain plan.
- Responsible for loading data into warehouse using Oracle Loader for history data.
- Responsible for moving the mappings and sessions from development repository to testing repository box.
- Troubleshooting production issues.
- Worked as a part of a team and provided 7 x 24 production support.
Environment: Informatica Power Center 9.5, Erwin, MS Visio, Oracle 11g, SQL, PL/SQL, Oracle Sql Developer Tool, SQL Server 2008, Flat Files, XML, Mainframe, Cobol Files, Control-M, UNIX Shell Scripting, Subversion.
Confidential
ETL Developer
Responsibilities:
- Involved in all phases of software development life cycle.
- Involved in requirements gathering and analysis to define functional specifications.
- Created various logical and physical data models interacting with the business team, leads and developers using ERWIN.
- Created technical specs documents for all the mappings using MS Visio 2010.
- Involved in extracting source data from Oracle, SQL Server 2000, DB2,Flat Files, VSAM files on different systems
- Developed mappings, sessions, workflows and workflow tasks based on the user requirement and scheduled the workflows in workflow manager and monitor the load status in the workflow monitor.
- Defined Target Load Order Plan for loading data into Target Tables.
- Worked with various look up cache like Dynamic Cache, Static Cache, Persistent Cache, Re Cache from database and Shared Cache.
- Used Informatica’s features to implement Type 1,2 changes in slowly changing dimension tables.
- Created and configured workflows, worklets & Sessions to transport the data to target systems using Informatica Workflow Manager.
- Used Mapplets and Reusable Transformations prevent redundancy of transformation usage and maintainability.
- Fine tuned the session performance using Session partitioning for long running sessions.
- Implemented performance tuning logic on Targets, Sources, mappings, sessions to provide maximum efficiency and performance.
- Created tables, views, primary keys, indexes, constraints, sequences, grants and synonym.
- Involved in developing optimized code using PL/SQL for Server related Packages to centralize the application through procedures, functions and packages.
- Created Oracle Stored Procedure to implement complex business logic for good performance and called from Informatica using Stored Procedure transformation
- Created Connected, Unconnected and Dynamic lookup transformation for better performance and increased the cache file size based on the size of the lookup data.
- Done various optimization techniques in Aggregator, Lookup, and Joiner transformation
- Implemented Informatica parameter files to filter the daily data from the source system.
- Developed Workflow Tasks like reusable Email, Event wait, Timer, Command, and Decision.
- Used Informatica debugging techniques to debug the mappings and used session log files and bad files to trace errors occurred while loading.
- Used pmcmd command in UNIX scripts to call Informtica session and workflows.
- Creating Test cases for Unit Test, System Integration Test and UAT to check the data .
- Used various Oracle Index techniques like B*tree, bitmap index to improve the query performance and created scripts to update the table statistics for better explain plan.
- Responsible for loading data into warehouse using Oracle Loader for history data.
- Responsible for moving the mappings and sessions from development repository to testing repository box.
- Troubleshooting production issues.
- Worked as a part of a team and provided 7 x 24 production support .
Environment: Informatica Power Center 9.5, Erwin, MS Visio, Oracle 11g, SQL, PL/SQL, Oracle Sql Developer Tool, SQL Server 2008, Mainframe, JCL, Dimension Tool, UNIX Shell Scripting, MKS Integrity.