Etl Developer Resume
MD
SUMMARY
- Extensive development/architecture experience with Talend, Pentaho & Informatica.
- Used different components in Pentaho like Database Lookup & Join, Generate rows, Calculator, Row normalizer & denormalizers, Java Script, Add constant, Add Sequence
- Experience in Data Warehouse development working with Data Migration, Data Conversion, and Extraction/Transformation/Loading using Pentaho Data Integration (Pentaho Kettle) with Oracle, SQL Server.
- Working experience on transforming data from various data sources to a Data warehouse using PDI 4.1/4.0 and Kettle 3.2 and interacting with the Business Analysts in identifying the needs and the requirements of the system.
- Highly experienced in creating PDI Jobs and Transformations in enterprise repositories according to requirement to integrate source data in to Target data ware house.
- Well versed with Talend ETL/BigData/ESB & Pentaho BigData components and used components like thdfsinput, thdfsoutput, thiveload to move the data from Hadoop to Hive.
- Installation and configuration of Talend Administration Center
- Development of typical data warehouse platform with integration between MS SQL Server and Oracle.
- Implementation of Change Data Capture components in Oracle environment Complex XML parsing Version control and best practices
- Used different components in talend MDM like tMDMInput, tMDMOutput, tFilterRow.
- Created complex mappings in Talend 5.2.2 using tMap, tJoin, tReplicate, tParallelize, tJava, tJavaFlex, tAggregateRow, tDie, tWarn, tLogCatcher, TESBInput, tESBrecive, tESBOutput, tRest, tSoap, etc.
- Used tStatsCatcher, tDie, tLogRow to create a generic joblet to store processing stats.
- Created Talend Mappings to populate the data into dimensions and fact tables.
- Broad design, development and testing experience with Talend Integration Suite & Talend MDM and knowledge in Performance Tuning of mappings.
- Proficient in supporting Data warehouse ETL activities using queries and functionalities of SQL, PL/SQL,SQL*Loader and SQL * Plus. Solid experience in implementing complex business rules by creating re - usable transformations and robust mappings/mapplets using various transformations like Unconnected and Connected lookups, Source Qualifier, Router, Filter, Expression, Aggregator, Joiner, Update Strategy etc.
TECHNICAL SKILLS
Primary Skills: Business Problem Analysis Database Architectural Design Data Modeling ETL & Reporting
ETL Tools: Pentaho Data Integration 4.0/4.1 Talend 5.x Informatica SSIS
Reporting Tools: Pentaho Report Designer 3.x Pentaho Analyzer Pentaho Schema Workbench Pentaho Analyzer Business Objects Crystal Reports
Database Platforms: SQL Server 2008/2005
Oracle 10g/9i/8i/8/7.x: MySQL 5.x
Analysis/Design Tools: MS Visio ER/Win E/R Studio
Languages: SQL & PL/SQL Unix Shell C/C++ Java / JavaScript
Operating Systems: Unix Linux Windows Server
PROFESSIONAL EXPERIENCE
Confidential, MD
ETL Developer
Responsibilities:
- Responsible for gathering information from clients regarding business requirements and collaborated with business technicians to research existing business and system processes.
- Interacted with business analysts and End client to understand technical and functional requirements for creating new Job.
- Developed Talend ETL jobs to push the data into Talend MDM and develop the jobs to extract the data from MDM.
- Developed InformaticaReusable Transformations(Joiner, Sorter, Aggregator, Expression, Lookup, Router, Filter, Update Strategy, Sequence Generator, Normalizer and Rank) and processing tasks usingWorkflow Managerto move data from multiple sources into targets.
- Written SQL Queries on Oracle Database to optimize the mappings in informatica.
- Developed the ETL mappings for XML, .csv, .txt sources and also loading the data from these sources into relational tables with Talend.
- Implementing Data Integration process with Talend Integration Suite 5.x
- Designing, developing and deploying end-to-end Data Integration solution.
- Used different components in talend like tmap, tmssqlinput, tmssqloutput, tfiledelimitede, tfileoutputdelimited, tmssqloutputbulkexec, tunique, tFlowToIterate, tintervalmatch, tlogcatcher, tflowmetercatcher, tfilelist, taggregate, tsort, tMDMInput, tMDMOutput, tFilterRow.
- Developed Scd Type1 & Type2 jobs with Talend SCD component and used tlogcatcher, tstatcatcher and tflowmeter to capture the statistics.
- Developed complex custom reports using Pentaho Report Designer which includes developing Cascading pick-lists, Drill-throughs, Hyperlinks, sub-reports etc, functionality into these reports.
- Developed several Pentaho Reports, Dashboards, XActions and Analyzer Reports for the client.
- Designed and deployed custom dashboards on Pentaho User Console.
- Integrated Pentaho reports and dashboards with the client’s existing front-end application and web portals.
- Used Pentaho Data Integration 4.0 for ETL extraction, transformation and loading data from heterogeneous source systems such as excel and flat files.
- Used PDI transformations to cleanse data for duplication, derived values and address parsing.
- Created complex PDI mappings to load the data warehouse. The mappings involved extensive use of transformations like Dimension Lookup/Update, Database Lookup & Join, Generate rows, Calculator, Row normalizer & denormalizers, Java Script, Add constant, Add Sequence etc.
- Extensively worked with enterprise Repositories, PDI Job Servers and Enterprise console.
- Responsible for Debugging and testing of PDI Jobs.
- Optimized data mappings to achieve faster load.
- Performed debugging and performance tuning of sources, targets and mappings.
- Worked with Parameters/Variables in PDI jobs and transformations to achieve automation.
- Created Java Scripts and worked with Conditional statements and While Loops to implement complex logic.
- Extensively used “Define Error Handling” to handle exceptions and wrote Scripts to automate the Job Process.
- Performance tuning of the SQL queries by restructuring the Joins, creation of required indexes to avoid table scans, error handling of queries etc.
- Loaded Data into Target using from flat files, XML and database tables as source.
- Performed migration of Data from Excel, Sybase, Flat file, Oracle, and MS SQL Server.
- Tested and Validated PDI ETL Jobs, monitored Daily PDI ETL schedules.
- Performed troubleshooting and provided resolutions to ETL issues.
Environment: Talend Integration Suite 5.5, Talend BigData, Talend MDM, Informatica, SQL Developer, Oracle 10g, SQL Server 2005/2008, Pentaho BI Suite (Data Integration Designer, Report Designer, Dashboard Designer, Analysis View, Pentaho Analyzer, Design Studio, Mondrian Server), SQL Management Studio 2008, SQL Server 2005 & 2008,Windows Server 2008, JavaScript
Confidential, Atlanta
ETL Developer
Responsibilities:
- Participated in Requirement gathering sessions and developed PDI job specifications based on information acquired from analysis of source data, user requirements and business Rules.
- Developed InformaticaReusable Transformations(Joiner, Sorter, Aggregator, Expression, Lookup, Router, Filter, Update Strategy, Sequence Generator, Normalizer and Rank) and processing tasks usingWorkflow Managerto move data from multiple sources into targets.
- Written SQL Queries on Oracle Database to optimize the mappings in informatica.
- Extracted the data using Pentaho Data Integration Designer (Kettle) from the flat files and other RDBMS databases like SQL Server 2005/2008 and Oracle 10g into staging area and populated onto Data warehouse.
- Developed Talend ETL jobs to push the data into Talend MDM and develop the jobs to extract the data from MDM.
- Developed the ETL mappings for XML, .csv, .txt sources and also loading the data from these sources into relational tables with Talend.
- Implementing Data Integration process with Talend Integration Suite 5.x
- Designing, developing and deploying end-to-end Data Integration solution.
- Used different components in talend like tmap, tmssqlinput, tmssqloutput, tfiledelimitede, tfileoutputdelimited, tmssqloutputbulkexec, tunique, tFlowToIterate, tintervalmatch, tlogcatcher, tflowmetercatcher, tfilelist, taggregate, tsort, tMDMInput, tMDMOutput, tFilterRow.
- Collaborated with Database Administrators, Developers, and Analysts on all aspects of data management.
- Used error handling strategy for trapping errors by defining error handling in all crucial steps of ETL transformations and sending errors to an error table.
- Monitored and performed troubleshooting batches and sessions for weekly and Monthly extracts from various data sources across all platforms to the target database.
- Involved in Performance tuning Confidential source, target, mappings, sessions, and system levels.
- Tested the data and data integrity among various sources and targets and was associated with Production support team in various performance related issues.
- Developed UNIX shell scripts to move source files to archive directory.
- Designed various types of reports using Pentaho Report Designer with Complex Formulas, Custom Functions, JavaScript Expressions, Sub Reports and Drill throughs by means of hyperlinks.
- Assisted in constructing reports UI with Java Eclispe SDK to include parameter values to be passed to reports.
- Designed the reports with multi level of groupings (hierarchies).
- Helped tune the database to resolve performance issues and ensured high performance through periodic performance evaluation and improvements.
- Involved in Unit, Integration, system, and performance testing levels.
- Interacted with executives to learn the business objectives and propose creative solutions to meet the objectives
- Provided Recommendations for cost mitigation solutions brought to the table with design approaches with the software development team to effectively represent the required data for analysis
- Documented for ERDs, Pentaho Report Templates and Pentaho Schema and Cube Designs.
Environment: Informatica, Talend Integration Suite 4.5, Talend MDM, Talend BigData, SQL Server 2005/2008, Oracle 10g, Pentaho BI Suite (Kettle, Report Designer, Design Studio, Pentaho Mondrian), SQL Management Studio 2008, Windows Server 2003/2008, JavaScript
Confidential, Cincinnati, OH
Informatica/Pentaho Developer
Responsibilities:
- Involved in gathering business requirements, functional requirements and data specification.
- Designed Data Warehouse and Database for the system using Star Schema which includes Tables, Stored Procedure and Indexes in SQL Server Management Studio.
- Developed InformaticaReusable Transformations(Joiner, Sorter, Aggregator, Expression, Lookup, Router, Filter, Update Strategy, Sequence Generator, Normalizer and Rank) and processing tasks usingWorkflow Managerto move data from multiple sources into targets.
- Used Data Viewers in SSIS Packages to see whether the data flow properly or not.
- Converted SSIS packages into Informatica ETL jobs.
- Created and updated complex Stored Procedures, Triggers and User Defined Functions for logical T-SQL implementation.
- Created Views to facilitate easy user interface implementation and Triggers on them to facilitate consistent data entry into the database.
Environment: MS SQL Server 2005 SQL Server Integration Services (SSIS), Pentaho Kettle Spoon Designer v3.2, SQL