Sr. Talend/ Etl Developer Resume
Nashville, TN
SUMMARY
- Over 9+ years of experience in Designing, Analysis, Development, Maintenance, Implementation of Relational Database (OLTP) and Data Warehousing Systems (OLAP) using Oracle, Teradata, Netezza, Informatica, Talend& Big Data.
- Analyzed, designed and developed Extraction, Transformation and Load (ETL) processes for Data Warehousing and Data Migration projects using Informatica Power Center including Designer, Repository Manager, Workflow Manager, Workflow Monitor and Administration Console.
- Extracted data from various source systems like Oracle, Mainframes, XMLand flat files as per the requirements.
- Extensive experience in writing and executing scripts for validation and testing of the sessions, data integrity between source and target database and for report generation.
- Developed Complex mappings from varied transformation logics like Unconnected/Connected lookups, Router, Filter, Expression, Aggregator, Joiner, Union, Update Strategy and more. Used debugger to test and fix mapping.
- Design & implement data quality & business rules using IDQ and Powercenter workflows, Mappings, Mapplets, Exception tables, ad - hoc reporting, Data quality score cards.
- Solid experience in designing ETL Jobs usingTalendOpen Studio (TOS) and Talend Integration suite.
- Handling Team in the absence of my seniors.
- Solid understanding knowledge of Ralph Kimball & Bill InmonMethodologies
- Experience in Data Analysis, ODS, Dimensional Data Modeling, Data Extraction Transformation & Loading (ETL), Building Cubes, Creating Meta layer, Data Mining, End User Reporting, Web based Reporting & Implementation, Creating Test scripts and user signoff documents.
- Experienced in the Integration of various data sources like Oracle, XML, DB2, COBOL files and Flat Files into the staging area, ODS, Data Warehouse and Data Mart.
- Experienced in Dimensional Data Modeling using Erwin, Slowly Changing Dimensions Type I & II, Star Schema /Snowflake Schema modeling, Fact & Dimensions tables, Physical&Logical data modeling.
- Experience in creating Ab Initio (or Datastage, Informatica or Talend) projects and sandboxes in lower life cycle environments, experience in configuration and support of multiple test environments.
- Extensively used Informatica Repository Manager and Workflow Monitor.
- Experienced in debugging mappings. Identified bugs in existing mappings by analyzing the data flow and evaluating transformations.
- Wrote complex SQL scripts to avoid Informatica Joiners, Unions and Look-ups to improve the performance as the volume of the data was heavy.
- Used Informatica Power Center Server manager to create sessions, batches to run Post SQL and Per SQL with the logic embedded in the mappings.
- Expertise in creating mappings, mapplets and reusable transformations using Informatica. Good knowledge in Scheduling ETL jobs like $U, Tidal, Autosys tools.
- Responsible for implementing all of the technical Data Warehousing solution to the client, in accordance with an agreed technical design.
- Expert in the field of design, development, and implementation of processes for Data Warehousing and Data Integration projects using InformaticaPowerCenter and PowerExchange.
- Expertise in performance tuning of SQL and ETL code and writing efficient and complex queries on huge volumes of data.
- Proficient in understanding business processes / requirements and translating them into technical requirements.
- Expert in designing and developing Unix Shell Scripts for automation of ETL Processes, Error handling and Auditing.
- Excellent implementation of data warehouse/data marts using various data warehouse concepts and principles like Change Data Capture, Slowly Changing Dimensions, Normalization/De-normalization (Star and snow-flake Schema Modeling).
- Excellent experience working on Slowly Changing Dimensions (SCD) Type 1, 2 and 3 to keep track of historical data.
- Solid experience in loading and maintaining Data Warehouses in Teradata &Netezza.
- Extensively used tools are MLoad, BTeq, FastExport and FastLoad to design and develop dataflow paths for loading transforming and maintaining data warehouse.
- Heavily used nzsql and nzloadNetezza utilities to load data from source to Netezza
- Good programming/debugging/troubleshooting skills in software development.
- Extensively used various Performance Tuning techniques to improve the session performance.
- Designed ETL architecture, created data flow diagrams, process flow diagrams, and high level ETL design.
- Designed and developed ETL job orchestration framework, CDC frameworkand Data Qualityframework.
- Worked on POCs to load data from Oracle to Hadoop to Hive.
- Loaded data into Hadoop (Hive) using sqoop. Created tables into Hadoop (Hive) out of a flat file.
- Solid experience in support and knowledge transfer to production team.
TECHNICAL SKILLS
Databases: Oracle 7/8.x/9.x/10g/11g/12c, SQL Server 2008/2012, Teradata, Netezza, DB2MS Access.
Development Tools: SQL Navigators, Toad, Stylus Studio (XML), XMLDB, Trans-SQL
ETL Tools: Talend 14/15, TIS, TOS, Informatica Power Center 9.5/9.1/8.6.1, IBM Data Stage.
Other Tools: XML Spy, Stylus studio, Visual Source Safe. Developer, Oracle Express, Teradata SQL Assistant, NetezzaAginity.
Languages: SQL, PL/SQL, WSDL, eScript, Siebel VB, Java Script, XML & XSD.
Operating System: UNIX (Sun Solaris), Windows 95/98/2000/NT/XP/7.
PROFESSIONAL EXPERIENCE
Confidential, Nashville,TN
Sr. Talend/ ETL Developer
Responsibilities:
- Acquire and interpret business requirements, create technical artifacts, and determine the most efficient/appropriate solution design, thinking from an enterprise-wide view.
- Involved in complete Software Development Lifecycle Experience (SDLC) from Business Analysis to Development,Testing, Deployment and Documentation.
- Worked with the Data Integration Team to perform data and application integration with a goal of moving moredata more effectively, efficiently and with high performance to assist in business critical projects coming up withhuge data extraction.
- Managed TalendMigration/Upgrade, maintaining existing Talend Jobs, improving/template current jobs, migrationof Major Talend Job to new process design, supporting data quality for auditing.
- Design and develop scalable data acquisition and transformation jobs using Talend open source or enterprise.
- Architecture and design support to provide solution for business initiated requests/ projects
- Used Teradata Utilities fastload, multiload, tpump to load data, Wrote BTEQ scripts to transform data
- Perform technical analysis, ETL design, development, testing, and deployment of IT solutions as needed by business or IT.
- Participate in designing the overall logical & physical Data warehouse/Data-mart data model and data architectures to support business requirements
- Creating ETL jobs usingTalendMDM tool to load and process the data as per requirements.
- Explore prebuiltETLmetadata, mappings and DAC metadata and Develop and maintain SQL code as needed for SQL Server database.
- Performed data manipulations using variousTalendcomponents like tMap, tJavarow, tjava, tOracleRow, tOracleInput, tOracleOutput, tMSSQLInput and many more.
- Provided Teradata Data Warehouse Administration and maintain existing physical and logical model
- Analyzing the source data to know the quality of data by usingTalendData Quality.
- Developed ETL jobs to extract the data from flat files, Transform the data and Load (ETL) it to flat file.
- Troubleshoot data integration issues and bugs, analyze reasons for failure, implement optimal solutions, and revise procedures and documentation as needed.
- Involved in Migration projects to migrate data from data warehouses on Oracle/DB2 and migrated those toTeradata, Netezza.
- Use SQL queries and other data analysis methods, as well as TalendEnterprise Data Quality Platform for profilingand comparison of data, which will be used to make decisions regarding how to measure business rules andquality of the data.
- Developed ETL jobs to validate & transform data that were ready to be loaded into Oracle database.
- Worked on TalendRTX ETL tool, develop jobs and scheduled jobs in Talend integration suite.
- Extensively used theconcepts of ETL to load data from AS400, flat files to Saleforce .
- Writing Teradata SQL queries to join or any modifications in the table
- UsedTalendreusable components like routines, context variable and globalMap variables.
- Responsible to tuneETLmappings, Workflows and underlying data model to optimize load and queryPerformance.
- DevelopedTalendESB services and deployed them on ESB servers on different instances.
- Involved in designing dimensional modeling and data modeling using Erwin tool.
- Implementing fast and efficient data acquisition using Big Data processing techniques and tools.
- Monitored and supported theTalendjobs scheduled throughTalendAdmin Center (TAC) Talend Developer
- Developed oracle PL/SQL, DDLs, and Stored Procedures and worked on performance and fine Tuning of SQL&
- Modified reports and Talend ETL jobs based on the feedback from QA testers and Users in development and
Environment: Talend, Teradata, Teradata SQL Assistant, Oracle 12c, IBM DB2, TOAD, BusinessObjects XI3.5, MLOAD, SQL Server 2012, XML, SQL, Hive, Pig, SQL, PL/SQL, Teradata, Netezza, TIS.
Confidential, NYC, NY
Sr. Talend/ETLDeveloper
Responsibilities:
- Worked closely with Business analysts and Data architects to understand and analyze the user requirements.
- Used Teradata utilities (TPT, BTEQ) to load data from source to target table.
- Created various kinds of indexes for performance enhancement.
- Performance tuned and optimized various complex SQL queries.
- Transform data to various sources using SQL Server Integration Service andTalendOpen Studio.
- Created ETL Mapping with TalendIntegrationSuite to pull data from Source, apply transformations, and load data into target database.
- Create ETL solution for requirement using map reduce (BIG data Hadoop/Pentaho) concept which extract data from SOR feed file and load data TDI hive tables.
- Delivered MDM stewardship and datagovernance program. Data monitoring &Notification processes are designed.
- Developed mappings /Transformation/Joblets and designed ETL Jobs/Packages using Talend Integration Suite (TIS) in Talend.
- Scheduled Pentaho ETL jobs and deployed in Pentaho BI Server in production environments.
- Created mappings using the transformations like Source Qualifier, XML Source Qualifier, Aggregator, Expression, Lookup, Router, Normalizer, Filter, Update strategy and Joiner transformations.
- ExcellentDataWarehousing concept Metadatamanagement (MDM)
- Implementing fast and efficient data acquisition using Big Data processing techniques and tools.
- Created Talend jobs to populate the data into dimensions and fact tables.
- Experience integrating the components of the following DB2,Data Stage (ETL).
- Developed and scheduled Workflows using task developer, worklet designer, and workflow designer in Workflow manager and monitored the results in Workflow monitor.
- Used Talendjoblet and various commonly used Talend transformations components like tMap, tDie, tConvertType, tFlowMeter, tLogCatcher, tRowGenerator, tSetGlobalVar, tHashInput&tHashOutput and many more.
- Created complex SCD type 1 & type 2 mappings using dynamic lookup, Joiner, Router, Union, Expression and Update Strategy Transformations.
- UtilizedSDLC and Agile methodologies such as SCRUM.
- Worked on Dimension as well as Fact tables, developed mappings and loaded data on to the relational database.
- Used Talend components tMap, tDie, tConvertType, tFlowMeter, tLogCatcher, tRowGenerator.
- Responsible for Performance Tuning at Talend level and SQL queries level.
- DevelopedTalendESB services and deployed them on ESB servers on different instances.
- Loading the data from different sources into relational tables withTalendETL.
- Developed complex Talend jobs mappings to load the data from various sources using different components.
- Wrote SQL overrides in source Qualifier in order to filter the data more effectively at the source level.
- Wrote Complex OLAP SQL's inTeradata.
- Provide ETL solution to the requirement using BIG DATA Hadoop/Pentaho.
- Designed and developed Big Data analytics platform for processing customer viewing preferences and social media comments using Java, Hadoop, Hive and Pig.
- Monitored and supported theTalendjobs scheduled throughTalendAdminCenter (TAC)
- Successfully migrated the mappings to the testing and production environment.
- Worked on both SAP MDM and Informatica.
- Created source table definitions in the DataStage Repository.
- Installation of Talend Open Studio (TOS) and Configuration along with Java JRE & JDK.
- Responsible for creating complete test cases, test plans, test data, and reporting status ensuring accurate coverage of requirements and business processes.
Environment: Talend, Erwin, Teradata, Oracle 11g, Visio, TOAD, SQL*Plus, XML, Windows, Winscp, UNIX, Putty, Control-M, Business Objects Xi3.1, SSIS, SSRS, Teradata SQL Assistant.
Confidential, Conway, AR
Sr. ETL Developer
Responsibilities:
- Responsible for Requirement Gathering Analysis and End user Meetings
- Worked on writing various stored procedures using PL/SQL on Oracle database to achieve complex functionality.
- Imported Source/Target Tables from the respective databases and created reusable transformations (Joiner, Routers, Lookups, Rank, Filter, Expression, Update, and Aggregator) in a Mapplet and created new mappings using Designer module of Informatica.
- Participated in the SIT testing and supported the System Acceptance and User Acceptance testing.
- Used DataStage as an ETL tool to extract data from sources systems, loaded the data into theORACLEdatabase.
- Worked on Informatica Power Center - Source Analyzer, Data warehouse designer, Mapping Designer &Mapplet, Transformations, Work Flow Manager (Task Developer, Worklets, and Work Flow Designer) and Work Flow Monitor.
- Worked onInformaticaData Quality (IDQ) to resolve customers address related issues.
- Involved with Informatica team members in Designing, document and configure the InformaticaMDMHub to support loading, cleansing, matching, merging, and publication of MDM data.
- Developed, tested and deployed full code package for EDW in SVN & Visual Source Safe.
- Developed ETL deployment plan for the production environment & provided run books to production support team.
- Using the Informatica and some Ab Initio development tool and SQL, support a large enterprise data warehouse used for statisticalmodeling.
- Extensively worked on ETL Informatica transformations effectivelyincluding - Source Qualifier, Connected - Unconnected Lookup, Filter, Expression, Router, Union, Normalizer, Joiner, Update, Rank, Aggregator, Stored Procedure, Sorterand Sequence Generator and created complex mappings.
- Worked on data cleansing and standardization using the cleanse functions in Informatica MDM.
- DevelopingIDQstandardization mappings according to rules and duplicates.
- Extensively used Change data capture concept in Informatica using mapping variables to capture the changes to the data warehouse.
- Used Teradata SQL Assistant, Teradata Administrator, PMON and data load/export utilities like BTEQ, FastLoad, Multi Load, Fast Export, Tpump on UNIX/Windows environments and running the batch process for Teradata.
- Extensively worked on developing Informatica Mappings, Mapplets, Sessions, Workflows and Worklets for data loads from various sources such as Oracle, ASCII delimited Flat Files, EBCDIC files, XML, COBOL, DB2, and SQL Server.
- Loaded data into Hadoop (Hive) using sqoop. Created tables into Hadoop (Hive) out of a flat file.
- Migrated various rules as well as mapplets developed atIDQDeveloper tool to the PowerCenter.
- Responsible for developing, testing, debugging, support and maintenance for the ETL processes using InformaticaPower Center.
- Involved in developing the data loading and extraction processes for big data analysis.
- Solid experience in performance tuning on TeradataSQL Queries and Informaticamappings.
- Worked extensively on the Master Data Management (MDM) and application used for MDM
- Implemented Proof of Concepts on Hadoop stack and different bigdataanalytic tools, migration from different databases like mysql, oracle to hadoop.
- Worked on Teradata SQL Assistant, Teradata administrator, Teradataview point and BTEQscripts.
- Used the DataStage Designer to develop processes for extracting, cleansing, transforming, integrating and loading data into staging tables.
- Published cubes and reports onto Pentahorepository and refreshed PentahoBI repository after uploading each object to be available for central use.
- Worked as Being a part of a POC effort to help build new Hadoop clusters.
- Created Operation Manual documents and developed and updated other technical documentations.
- Removing duplicates, standardizing data (mass maintaining), and incorporating rules to eliminate incorrect data from entering thesystem in order to create an authoritative source of master data Using MDM.
- Used AbInitio’s DQE for data quality solution for enterprise-level data processing and data management systems.
- Extensively used various active and passive transformations like Filter Transformation, Router Transformation, Expression Transformation, Source Qualifier Transformation, Joiner Transformation, and Look up Transformation, Update Strategy Transformation, Sequence Generator Transformation, Rank Transformation, and Aggregator Transformation.
- Responsible for best practices like naming conventions, Performance tuning, and Error Handling
- Responsible for Performance Tuning at the Source level, Target level, Mapping Level and Session Level
- Solid Expertise in using both Connected and Un connectedLookuptransformations
- Extensively worked with various lookup caches like Static Cache, Dynamic Cache, and Persistent Cache.
- Worked on writing various stored procedures using PL/SQL on Oracle database to achieve complex functionality.
- Worked on creating various mapplets in Informatica to provide re-usability.
- Responsible for best practices like naming conventions, Performance tuning, and Error Handling.
- Involved in defining the overall strategy for design and standards by creating many checklists for smooth deployments.
- Developed Slowly Changing Dimension Mappings for Type 1 SCD and Type 2 SCD
- Worked on extracting real time data and loading to data warehouse.
- Scheduling and performance tuning of Informatica mappings.
- Worked closely with the project team to formulate and implement a flexible system design that meets functional requirements.
- Created data models diagrams using Visio and Erwin for EDW and datamarts.
- Collaborated with data modelers, ETL developers in the creating the Data Functional Design documents.
- Created the Staging mappings for delta detection using the different Transformations (Expression, Lookup, Joiner, Filter, Router, Stored Procedure etc.)
Environment: Informatica on UNIX, Teradata R13, IDQ, Netezaa, Oracle 11g, PL/SQL, TOAD, UNIX, FASTLOAD, MULTILOAD, SSIS, TPUMP, TPT
Confidential, Princeton, NJ
Sr. ETL Developer/Talend Developer
Responsibilities:
- Worked on using TalendIntegration suite and created many jobs on Talend
- Utilized Talend components such as tFileExist, tFileInputFullRow, tFileList, tFileCopy, tLogRow, tDie, tMap etc. and created various jobs in Talend
- Parsed high-level design specification to simple ETL coding and mapping standards.
- Developed complex Talend jobsmappings to load the data from various sources using different components.
- Performed match/merge and ran match rules to check the effectiveness of MDM process on data.
- Experience in CodingTeradataSQL,TeradataStoredProcedures, Macros and Triggers
- Responsible for developing ETL jobs to load the various data marts using Datastage.
- Loading data from various data sources and legacy systems into Teradata production and development warehouse using BTEQ, FASTEXPORT, MULTI LOAD, FASTLOAD and Talend
- Skewed redistributions, join order, optimizer statistics, physical design considerations (PI, NUPI and USI, NUSI and JI etc) etc. In-depth knowledge of Teradata Explain and Visual Explain to analyze and improve query performance.
- Prepared Design Specification Documents.
- UsedTeradataload utilities like BTEQ, FLOAD, MLOAD and FXPORT.
- Involved in understanding Business and data needs and analyze multiple data sources and document data mapping to meet those needs.
- Worked with Teradata data lake
- Converted the existing Oracle materialized and relational views intoTeradataviews.
- Designed and developed a new ETL process to extract and load accounts from Legacy System by using theTalenddataintegration tool.
- Designed the Business Process for MDM Job by UsingTalendMDM.
- CreatedTalendjobs (ETL) to migrate data from heterogeneous sources such as MS Excel, Flat Files, .CSV files, Oracle etc to target databases. Also converted many SSIS packages toTalendJobs.
- Transform data to various sources using SQL ServerIntegration Service andTalendOpenStudio.
- Used Type 1 SCD and Type 2 SCD mappings to update slowly Changing Dimension Tables.
- Responsible for Performance Tuning at Talend level and SQL queries level
- Prepared Detail design documentation thoroughly for production support department to use as Hand guide for future production runs before code gets migrated.
- Responsible for design and developing TeradataBTEQ scripts, MLOAD based on the given business rules and design documents
Environment: Talend, Oracle 11g, SQL, PL/SQL, Toad, Putty, UNIX, Shell scripting, AIX, Crontab, Teradata, SSIS,Erwin.
Confidential, Cleveland, OH
Sr. ETL Developer
Responsibilities:
- Involved in business analysis and technical design sessions with business and technical staff to develop requirements document and ETL design specifications.
- Extracted data from various source systems like flat files and Oracle databases and to schedule the workflows.
- Designed Oracle based WellView Sources to Targets mappings using Informatica Developer Tool for Data cleansing, validating, integrating and matching using Informatica Data QualityIDQ.
- Created stored procedures to extract data from flat files.
- Wrote complex SQL queries on Netezza and used them in lookup SQL overrides and Source Qualifier overrides.
- Hands on experience using query tools like TOAD, SQL Developer, PLSQL developer, Teradata SQL Assistant and Query man.
- Handle slowly changing dimension (SCD) by using change capture stages and use data stage sub-version control to keep track of multiple versions of data stage jobs.
- Expertise in writing large/complex queries using SQL.
- Experience in developing ELT scripts in UNIX/NZSQL for aNetezzawarehouse.
- Wrote various UDF functions on Netezza.
- Developing the ETL mappings for XML, .CSV, .TXT sources and also loading the data from these sources into relational tables withTalendETL.
- DevelopingTalendjobs for Star and Snow-Flake schema models.
- Excellent Experience with different indexes (PI, SI, JI, PPI) and Collect Statistics.
- Loading the data into Netezza from legacy systems and flat files using complex UNIX scripts.
- Performance tuning, including collecting statistics, analyzing explains & determining which tables needed statistics. Increased performance by 35-40% in some situations.
- Identified potential bottlenecks with queries from the aspects of query writing,
- Extracted data from various sources like Oracle, Netezza and flat files and loaded into the target Netezza database.
- Created validation and error calculation mapplets usingIDQTool and migrated them to PowerCenter.
- Involved in Performance Tuning of ETL jobs by tuning the SQL used in Transformations.
- Designed and Developed Mappings with Reusable Transformations and Mapplets considering Functional and Non-Functional Requirements.
- Developed UNIX Shell scripts in conjunction with NZSQL/NZLOAD utilities to load data from flat files to Netezza database.
- Developed Complex Mappings for Data Integration based on Business Requirement and Logic.
- Involved in Unit Testing, Integration and User Acceptance Testing of mappings.
Environment: Informatica Power Center 8.6.0 / 7.1.1, Informatica Power Connect / Power IDQ9 / 8.x, Exchange for web services & MQ, Oracle, PL/SQL, UNIX Shell Script, Toad, SSIS, SSRS, Sql Server Windows, MS Access, Excel, VBA.
Confidential, Washington, DC
ETL Developer
Responsibilities:
- Developed InformaticaMappings to transform and load data into Oracle.
- Worked closely with the ETLLead, Data Modeler, Business Analysts to understand business requirements, providing expert knowledge and solutions on Data Warehousing.
- Used most of the transformations such as the Source qualifier, Router, Filter, Sequence Generator, Expression, Union, Joiner, Dyanamic Lookup etc as per the business requirement.
- UsedTeradataload utilities like BTEQ, FLOAD, MLOAD and FXPORT.
- Implemented performance tuning and optimization techniques.
- Interacted with Business analysts to collect the Business requirements and understand the usage.
- Developed Coding Standards and Best Practices, Technical Specification Templates, Source to Target Mapping Templates, Initial and Incremental load strategies.
- Involved in designing the data mart as per the reporting requirements with Type2 and Junk as dimension tables along with fact tables.
Environment: Informatica, Oracle, SQL, Toad, Unix(Shell Scripting)