Sr. Etl/ Talend Developer Resume
Lombard, IL
SUMMARY:
- 8+ years of strong experience in Analysis, Design and Development of Business Intelligence Solutions in Data Warehousing using Talend 6.2 and Informatica PowerCenter 9.5/9.1/8.6/8.1 as an ETL tool on Windows and UNIX based operating systems.
- Application Using ETL tools like Talend 6.3 and IBM Infosphere Data Stage 9.1.
- Expertise in Integration Services (SSIS) and SQL Server Reporting Services (SSRS) with Good knowledge in SQL Server Analysis Services (SSAS) • Exposure of ETL methodology for supporting Data Extraction, Transformation and Loading process in a corporate - wide ETL solution using Talend Open Source for Data Integration 6.2.
- Experience in developing ETL mappings, transformations and implementing source and target definitions in Talend.
- 4+ years of experience with Talend Open Studio (6.x/5.x) and 3+ years of experience with Talend Admin Console (TAC)
- Experience in working with Data Warehousing Concepts like OLAP, OLTP, Star Schema, Snow Flake Schema, Logical Data Modelling, Physical Modelling, Dimension Data Modelling
- Significant experience with Data Extraction, Transformation and Loading (ETL) from disparate data sources such as multiple relational databases and also worked on integrating data from flat files, CSV files and XML files into a common reporting and analytical data model.
- Extensive experience in Performance Tuning of SQL queries & Views
- Development of automated daily, weekly and monthly system maintenance tasks such as database backup, database integrity verification and index updates
- Experience in using AWS cloud components and connectors to make API calls for accessing data from cloud storage (Amazon S3, Redshift) in Talend Open Studio.
- Experience in Service oriented development using Talend ESB.
- Strong understanding of RDBMS concepts and experience in writing PL/SQL and SQL statements in databases.
- Extensively created mappings in Talend using t-Map, t-Join, t-Replicate, t-Parallelize, tJava, t-Java row, t-Die, t-Aggregate Row, t-Warn, t-Log Catcher, t-Filter, t-Global map etc.
- Highly proficient in the integration of various data sources involving multiple relational databases like Oracle, MS SQL Server, Teradata, DB2 and non-relational sources like COBOL Files and Flat File.
- Strong understanding of the principles of DW using fact tables, dimension tables and star/snowflake schema modeling.
- Experience in DI, MDM and ESB.
- Experience of Hadoop Architecture and various components such as HDFS, Name Node, Data Node, Job Tracker, Task Tracker, YARN and Map Reduce.
- Experience on cloud configuration in Amazon web services (AWS).
- Experience with the Architecture SOA, TOGAF and MDM modeling.
- Expertise on Exception Handling Mappings for Data Quality, Data Cleansing and Data Validation.
- Experience in Trouble shooting and implementing Performance tuning at various levels such as Source, Target, Mapping, Session and System in ETL Process. Have heavily worked on performance tuning long running Informatica mappings using Pushdown optimization and session partitioning.
- Strong Data Analysis and Data Profiling background using Informatica Analyst, Informatica Data Explorer (IDE) and also data cleansing background using Informatica Data Quality (IDQ).
- Experience in working with Standardizer, Parser, Match and Merge & Consolidation Transformations using IDQ.
- Worked with SQL/PL-SQL to write Complex SQL queries, Stored Procedures, Triggers, Functions & PL/SQL packages.
- Experience in Installing and Configuring Talend Administrator Centre, JobServe and Talend Runtime server.
- Create User, Projects, Configure SVN/GIT repositories, Nexus repository and Command Line in TAC.
- Experience on Scheduling Talend Jobs by using third party utilities like Crontab, Windows Scheduler etc.
- Experience on Scheduling Talend Jobs by using Job Conductor and Execution Plan in TAC Server.
- Configuring mail notification in TAC server to send error notification to users.
- Enabled log4j to capture exceptions in Talend Job.
- Experience on creating Audit tables to capture details of the Talend Job like Inserted rows, Updated rows, Job start time, End time etc.
TECHNICAL SKILLS:
ETL Tools: Talend Bigdata 6.3/5.6/5.1 Informatica Power Centre, IBM Infosphere DataStage.
BI Tools: MicroStrategy, IBM Cognos.
Front End Tools: Microsoft Project, Microsoft Office, Visible Analyst
Databases: Hive, Impala, Mongo DB, Teradata, Oracle, Netezza, Oracle Exadata, SQL Server, DB2, Access etc.
Document management: Visual Source Safe 6.0, Share point, Ultra Edit, Documentum
Defect management tool: HP Quality centre 10.0
Programming Languages: HTML, SQL, PL/SQL, Core Java, Unix
Data modeling: Power Designer, Erwin, ER Studio, MS Visio
Code repository: SVN, GIT, Bitbucket.
Scheduling tools: AutoSys, control.
PROFESSIONAL EXPERIENCE:
Confidential, Lombard, IL
Sr. ETL/ Talend Developer
Responsibilities: -
- Based on the requirement for knowing type of feed, source file format, business and transformations rules required for the process.
- Developed jobs with bigdata components like HDFS and HIVE components to capture raw data into Hadoop system.
- Designed partial restart ability of workflows and logging mechanism.
- Ingested the data from various Databases into HDFS by SQOOP tool.
- Extensively used Big Data components such as tSqoopExport, tSqoopImport, tHDFSInput, tHDFSOutput, tHiveLoad, tHiveInput, tPigLoad, tPigFilterRow, tPigFilterColumn, tPigStoreResult, tHbaseInput, tHbaseOutput along with executing the jobs in Debug mode and also utilizing the tlogrow component to view the sample output.
- Created Hive tables, partitions and implemented incremental imports to perform ad-hoc queries on structured data.
- Developed jobs to move inbound files to HDFS file location based on monthly, weekly, daily and hourly partitioning and also developed jobs for moving the outbound files to different locations.
- Loaded data from various sources, pre/post-processing using Hive and created tables in our cluster.
- Also utilized various data integration components in Talend like tmap, tMssqlinput, tmssqloutput, tfiledelimitede, tfileoutputdelimited, tmssqloutputbulkexec, tunique, tFlowToIterate, tintervalmatch, tlogcatcher, tflowmetercatcher, tfilelist, taggregate, tsort,tMDMInput, tMDMOutput, tFilterRow.
- Worked on tFileInputPositional, tFlowToIterate, tMap, tFilterRow, tUniqueRow components to validate, extract and convert the data from the positional file.
- Configured context variables, global variables for the conversion using Java code for some custom processing.
- Worked with main, iterate, reject, OnSubjobOk, OnComponentOK flow control components.
- Extensively used Traces/Java Debug to debug the errors and analyze the flow.
- Responsible for Build and deployment of all the jobs from TAC.
- Responsible for scheduling monitoring and troubleshooting workflows.
- Published hive views above the hive base tables as per the business logic for the downstream systems to pull data from Hadoop.
- Developed automated workflows for monitoring the landing zone for the files and ingestion into HDFS in Bedrock Tool and Talend.
- Used HBase for storing the Metadata of files and maintaining the file patterns. Environment:Talend Platform for Big Data 6.3, UNIX, Oracle 10g, Oracle, TAC, cloudera, Hive, Impala, datalake, bitbucket, AutoSys, service now, JIRA.
Confidential, Farmington, CT
Sr. ETL/ Talend Developer
Responsibilities: -
- Interacted with business team to understand business needs and to gather requirements.
- Designed target tables as per the requirement from the reporting team and designed Extraction, Transformation and Loading (ETL) using Talend.
- Created Technical Design Document from Source to stage and Stage to target mapping.
- Worked with Talend Studio (Development area) &Admin Console (Admin area).
- Created Java Routines, Reusable transformations, Joblets using Talend as an ETL Tool.
- Created Complex Jobs and used transformations like tMap, tOracle (Components), tLogCatcher, tStatCatcher, tFlowmeterCatcher, File Delimited components and Error handling components (tWarn, tDie).
- Identified performance issues in existing sources, targets and Jobs by analyzing the data flow, evaluating transformations and tuned accordingly for better performance.
- Manage all technical aspects of the ETL Jobs process with other team members.
- Consume data from XML files and convert data in to flat files.
- Extensively used XML components like tfileinputxml, textractxmlfield, fileoutputxml, twritexmlfield, txmlmap.
- Worked with Parallel connectors for parallel processing to improve job performance while working with bulk data sources.
- Worked on generating the web services classes by using Service Oriented Architecture SOA, WSDL, UDDI, and SOAP.
- Developed mappings to load Fact and Dimension tables, SCD Type 1 and SCD Type 2 dimensions and Incremental loading.
- Created contexts to use the values throughout the process to pass from parent child to child jobs and child to parent jobs.
- Worked on Joblets (reusable code) & Java routines in Talend.
- Performed Unit testing and created Unix Shell Scripts and provided on call support.
- ScheduleTalend Jobs using Job Conductor (Scheduling Tool in Talend) - available in TAC.
- Retrieved data from Oracle and loaded into SQL Server data Warehouse.
- Created many complex ETL jobs for data exchange and to Database Server and various other systems including RDBMS, XML, CSV, and Flat file structure.
- Created and reviewed scripts to create new tables, views, queries for new enhancement in the applications using TOAD.
- Monitoring the Data Quality, generating weekly/monthly/yearly statistics reports on production processes - success / failure rates for causal analysis as maintenance part and Enhancing existing production ETL process.
- Development of high-level data dictionary of ETL data mappings and transformations from a series of complex Talend data integration jobs.
Environment: Talend data integration 5.6.2, UNIX, flat files, XML, JSON, Oracle Exadata, SQL developer, Putty, Unix scripting, service now, control M, JIRA, Git hub, Agile.
Confidential, Irvine, CA
Sr. ETL/ Talend Developer
Responsibilities:
- Involved in designing an ETL job to validate a Flat File API using tschema check compliance component and checking the data integrity using tmap component against list of static values provided by the client.
- Improved the performance of long running jobs by updating the source SQL queries.
- Maintained client relationship by communicating the daily statuses and weekly statuses of the project.
- Integrating the customer master data from SAP CRM to Netezza.
- Extensively used the crm components like tSAPBapi and XML components for consuming IDOCS.
- In addition to Development worked as Talend admin: Creating Projects/ Scheduling Jobs / Migration to Higher Environments.
- Actively participated in all phases of System Development Life Cycle (SDLC) starting from requirement gathering all the way up to deployment in production and system testing.
- Extensive experience with Talend MDM to perform data quality and data profiling.
- Involved in Data Extraction from various Databases & Files using Talend data integration.
- Extensively Used tmap component which does lookup & Joiner Functions
- Involved in creating context variables in repository and use these context variables to run multiple jobs in Talend.
- Used more components in Talend such as: tjava, tschemacompliancecheck, tmap, toracleinput, tdelimitedinputfile, taddCRCRow, tpostgresqlrow, tpostgresqloutputbulk, components etc. in many of my Jobs Design.
- Created projects in Talend administration center (TAC)and assigned roles to the users.
- Worked on Joblets (reusable code) & Java routines in Talend.
- Mainly Involved in Performance Tuning of long running ETL jobs.
- Involved in creating Email handling task in each ETL job, sending failure notification to the support team in case of job failure using tLogCather and tsendMail components from the palette.
- Implemented Error Logging, Error Recovery, and Performance Enhancements for various Application teams.
- Experience in using Repository Manager for Migration of Source code from Lower to higher environments.
Environment: Talend 5.1, SAP CRM, Netezza, Oracle 11g, SAP HANA, Flat Files, SQL developer, Putty, WinSCP 5.9, JIRA, Git hub, Unix scripting, Agile, TAC.
Confidential, Monroe, LA
ETL/ DataStage Developer
Responsibilities:
- Interacted with business users and performed Gap Analysis of Client requirements
- Elicit business solutions to develop SOX compliance into the existing procurement workflow executed through IT channels.
- Performed Analysis of Reference Data, Data sets and Asset classes in order to bring data into the central repository.
- Worked extensively with MS Visio and Rational tools to document requirements.
- Created the AS-IS and TO-BE process flow diagrams.
- Defined the scope of the project based on the gathered business requirements including documentation of constraints, assumptions, business impacts, project risks and scope exclusions
- Defined functional business requirements, analyzed client requirements, and generated workflow processes
- Prepared Business Requirement Document and then converted business requirements into Functional Requirements Specification.
- Assisted in project management activities like planning, development, task allocation, project execution Provided full project life cycle flowcharting, analysis and documentation.
- Responsible for extracting, transforming, and loading data from ODS, flat files and placing it into data warehouse.
- Designed, developed, and tested the jobs, using Data Stage Involved in some part of performance tuning.
- Developed simple and complex jobs using Data Stage to load dimensions and fact tables as per STAR schema techniques.
- Created Data Definitions and help maintain Metadata repository.
- Utilized Erwin for reverse engineering the database from DDL scripts.
- Experience in using MicroStrategy in creating and managing reports for an organization. Created documentation, worked with users for new module enhancements and testing.
- Interacted with the End users on a regular basis for resolving issues pertaining to the reports.
- Defined project milestones, schedules, monitoring progress and updating plans using MS Project.
- Prepared monthly status reports for Enhancement Requests to the project team that was used in efficient tracking and monitoring of open issues of the project.
Environment: Oracle, Toad, Data Stage, MicroStrategy, Autosys, UNIX, Microsoft (Word, Access, Excel, Outlook), MS-VISIO, MS-Project.
Confidential
ETL/ Informatica Developer
Responsibilities:
- Prepared ETL process flow documents based on the present process flow and business functionalities.
- Created mappings with heterogeneous sources like flat tiles, oracle databases and created targets in oracle using Informatica Mapping Designer.
- Developed Mappings/Workflows/Scheduling ETL process.
- Frequently using import and export utility to migrate session from developer’s folder to subject folder.
- Developed Re-usable piece of code like Mapplet to use across various modules.
- Used Power Exchange interface to extract the legacy data.
- Developed UNIX Shell Scripts to schedule the Informatica jobs.
- Involved in design changes specific to releases.
- Designed mapping templates to specify high-level approach.
- Extensively worked with Informatica components like Source Analyzer, Warehouse Designer, Transformation developer, Mapplet Designer, Mapping Designer, Repository manager, Workflow Manager, Workflow Monitor, Repository server and Informatica server to load data from flat files, SQL Server.
- Designed the mappings between sources (files and databases) to operational staging targets.
- Used Aggregator, sequence, look up, expression, filter, Joiner, Rank, Router, Sequence generator, Update Strategy transformations in populating the data process.
- Designed and developed the Informatica workflows/sessions to extract, transform and load the data into oracle Server.
- Worked with different Informatica tuning issues and fine-tuned the transformations to make them more efficient in terms of performance.
Environment: Informatica Power Center 8.6.x, Informatica Power exchange 8.6.x, Oracle 10g, SQL*Loader, UNIX, Toad