Sr. Etl/ Talend Developer Resume
Chicago, IL
SUMMARY:
- 8+ years of experience in IT Industry involving Software Analysis, Design, Implementation,Coding, Development, Testing and Maintenancewith focus on Data warehousing applicationsusing ETL tools like Talend6.3 andIBM InfoSphere DataStage 9.1.
- 3+ years of experience using Talend Integration Suite (6.x/5.x) / Talend Open Studio (6.x/5.x) and 2+ years of experience with Talend Admin Console (TAC).
- Experience working with Data Warehousing Concepts like Kimball/Inmon methodologies,OLAP, OLTP, Star Schema, Snow Flake Schema, Logical/Physical/ Dimensional Data Modeling.
- Experienced in working with Horton works distribution of Hadoop, HDFS, MapReduce, Hive, Impala, Sqoop, Flume, Pig, HBase, and MongoDB
- Experience in dealing with structured and semi - structured data in HDFS.
- In depth understanding of the Gap Analysis i.e., As-Is and To-Be business processes andexperience in converting these requirements into Technical Specifications andTest Plans.
- Highly Proficient in Agile, Test Driven, Iterative, Scrum and Waterfall software development lifecycle.
- Extensively used ETL methodology for performing Data Profiling, Data Migration, Extraction,Transformation and Loading using Talend and designed data conversions from wide variety ofsource systems including Netezza, Oracle, DB2, SQL server, Teradata, Hive, non-relational sources like flat files, XML and Mainframe Files.
- Experience in analyzing data using HiveQL and Pig Latin in HDFS.
- Extracted data from multiple operational sources for loading staging area, Data warehouse, DataMarts using SCDs (Type 1/Type 2/ Type 3) loads.
- Extensively created mappings in TALEND using tMap, tJoin, tReplicate, tParallelize,tConvertType, tflowtoIterate, tAggregate, tSortRow,tFlowMeter,tLogCatcher,tRowGenerator, tNormalize, tDenormalize, tSetGlobalVar, tHashInput, tHashOutput, tJava,tJavarow, tAggregateRow, tWarn, tLogCatcher, tMysqlScd, tFilter, tGlobalmap, tDie etc.
- Extensive experience in using Talend features such as context variables, triggers, connectorsfor Database and flat files like tMySqlInput, tMySqlConnection, tOracle, tMSSqlInput,TMSSqlOutput, tMSSqlrow, tFileCopy, tFileInputDelimited, tFileExists.
- Experience in using cloud components and connectors to make API calls for accessing data fromcloud storage (Google Drive, Salesforce, Amazon S3, DropBox) inTalend Open Studio.
- Experience in creatingJoblets in TALEND for the processes which can be used in most of the jobsin a project like to Start job and Commit job.
- Experience in monitoring and scheduling using AutoSys, Control M& Job Conductor (TalendAdmin Console) and using UNIX (Korn& Bourn Shell) Scripting.
- Expertise in creating sub jobs in parallel to maximize the performance and reduce overall jobexecution time with the use of parallelize component of Talend in TIS and using the MultithreadedExecutions in TOS.
- Experienced in creating Triggers on TAC server to schedule Talend jobs to run on server.
- Strong experience in Extraction, Transformation, loading (ETL) data from various sources into
- Data Warehouses and Data Marts using DataStageversions 9.1, 8.7, 8.1.
- Designed jobs in DataStage 9.1,8.7 using stages like Transformer, sequential, Aggregator, Data Set, File Set, Remove Duplicates, Sort, Join, Merge, Lookup, Funnel, Copy, Modify, Filter, Change Data Capture, Surrogate Key, External Source, External Target, Compare and Schedule jobs through UNIX shell scripts.
- Strong hands on experience using Teradata utilities like SQL Assistant, B-TEQ, FastLoad,MultiLoad, Fast Export, and TPUMP.
TECHNICAL SKILLS:
BigData: HDFS, Hive, Pig, Spark, HBase
Data Warehousing: Talend Open Studio (TOS) for Data Integration 6.3, InfoSphere DataStage 9.1, 8.7
Databases: Netezza, Teradata & utilities (BTEQ, FASTLOAD, FASTEXPORT, MULTILOADTRUMP),Oracle12c/11x/10g, DB2, Microsoft SQL Server, Hive, Impala, Sybase.
Programming: T-SQL, PL/SQL, HTML, XML.
Environment: Windows, UNIX (SunSolaris10,HP,AIX) & Linux
Scripting: Korn shell script & Windows batch scripting, JavaScript
Languages: SQL Developer, AginityWorkBench, Teradata SQL Assistant, SQL*Plus, Toad.
OtherTools: SQL Navigator, Putty, MS-Office, VMWare Workstation.
PROFESSIONAL EXPERIENCE:
Confidential - Chicago, IL
Sr. ETL/ Talend Developer
Responsibilities:
- Worked with Data mapping team to understand the source to target mapping rules.
- Analyzed the requirements and framed the business logic and implemented it using Talend.
- Involved in ETL design and documentation.
- Analyzed and performed data integration using Talend open integration suite.
- Worked on the design, development and testing of Talend mappings.
- Created ETL job infrastructure using Talend Open Studio.
- Load and transform data into HDFS from large set of structured data /Oracle/Sql server using Talend Big data studio.
- Used Big Data components (Hive components) for extracting data from hive sources.
- Wrote HiveQL queries using joins and implemented in tHiveInput component.
- Utilized Big Data components like tHiveInput, tHiveOutput, tHDFSOutput, tHiveRow, tHiveLoad, tHiveConnection, tOracleInput, tOracleOutput, tPreJob, tPostJob, tLogRow.
- Worked on Talend components like tReplace, tmap, tsort and tFilterColumn, tFilterRow,tJava, tjavarow, tConvertType etc.
- Used Database components like tMSSQLInput, tMsSqlRow, tMsSqlOutput, tOracleOutput, tOracleInput etc.
- Worked with various File components like tFileCopy, tFileCompare, tFileExist, tFileDelete, tFileRename.
- Worked on improving the performance of Talend jobs.
- Created triggers for a Talend job to run automatically on server.
- Worked on Exporting and Importing of Talend jobs.
- Created jobs to pass parameters from child job to parent job.
- Exported jobs to Nexus and SVN repository.
- Implemented update strategy on tables and used tJava, tJavarow components to read data from tables to pull only newly inserted data from source tables.
- Observed statistics of Talend jobs in AMC to improve the performance and in what scenarios errors are causing.
- Created Generic and Repository schemas.
- Developed project specific Deployment job responsible to deploy Talend jar files on to the windowsenvironment as a zip file, later, this zip file is unzipped and the files are again deployed to the UNIX box.
- Also, this deployment job is responsible to maintain versioning of the Talend jobs that are deployed in the UNIX environment.
- Developed shell scripts in UNIX environment to support scheduling of the Talend jobs.
- Monitored the daily runs, weekly runs and adhoc runs to load data into the target systems.
Environment: Talend 6.3/6.1, Oracle, IBM DB2, Teradata, HDFS, Hive, Impala, SQL, PL/SQL, HP ALM, JIRA.
Confidential, Atlanta
Sr. ETL/Talend Developer
Responsibilities:
- Worked in the Data Integration Team to perform data and application integration with a goal of moving more data more effectively, efficiently and with high performance to assist in business-critical projects coming up with huge data extraction.
- Perform technical analysis, ETL design, development, testing, and deployment of IT solutions as needed by business or IT.
- Participate in designing the overall logical & physical Data warehouse/Data-mart data model and data architectures to support business requirements
- Explore prebuilt ETL metadata, mappings and DAC metadata and Develop and maintain SQL code as needed for SQL Server database.
- Performed data manipulations using various Talend components like tMap, tJavarow, tjava, tOracleRow, tOracleInput, tOracleOutput, tMSSQLInput and many more.
- Analyzing the source data to know the quality of data by using Talend Data Quality.
- Troubleshoot data integration issues and bugs, analyze reasons for failure, implement optimal solutions, and revise procedures and documentation as needed.
- Worked on Migration projects to migrate data from data warehouses on Oracle/DB2 and migrated those toNetezza.
- Used SQL queries and other data analysis methods, as well as Talend Enterprise Data Quality
- Platform for profiling and comparison of data, which will be used to make decisions regarding how to measure business rules and quality of the data.
- Writing Netezza SQL queries to join or any modifications in the table
- Used Talend reusable components like routines, context variable and globalMap variables.
- Responsible to tune ETL mappings, Workflows and underlying data model to optimize load and Query performance.
- Developed Talend ESB services and deployed them on ESB servers on different instances.
- Implementing fast and efficient data acquisition using Big Data processing techniques and tools.
- Monitored and supported the Talend jobs scheduled through Talend Admin Center (TAC).
- Developed Oracle PL/SQL, DDLs, and Stored Procedures and worked on performance and fine
Environment: Talend 6.1/ 5.5.2, UNIX, Shell script, SQL Server, Oracle, Business Objects, ERwin, SVN.
Confidential - Chicago, IL
DataStage/TalendETL Developer
Responsibilities:
- Work with the Business team to better understand the requirements and determine the appropriate data source and conversion approach.
- Developed DataStage Jobs, define job parameters, range lookups.
- Extensively used the CDC (Change Data Capture) stage to implement the slowly changing Dimensional and Fact tables.
- Used stages like Transformer, sequential, Aggregator, Data Set, File Set, Remove Duplicates, Sort, Join, Merge, Lookup, Funnel, Copy, Modify, Filter, Change Data Capture, Surrogate Key, External Source, External Target, Compare and Schedule jobs through UNIX shell scripts.
- Assist the functional teams by insuring good functional requirements exist, converting resulting functional specs into technical specs for data integration.
- Created the DataStage jobs to load data from ECOM database to ODS to Business Intelligence layer
- Performed performance tuning of the jobs by interpreting performance statistics.
- Create DataStage components efficiently and made them reusable.
- Designing XML schema definitions (XSDs), extract and load to huge XML files.
- Involved in loading data from Oracle database to Teradata tables using DataStage.
- Design complex DataStage jobs with Java, DB2 extractions, SCD functionality and bulk loads.
- Created Jobs using different stages like Aggregators, Joins, Merge, Lookup, Source dataset, Row generator, Change Capture, Peak stages, Column generator, Oracle, Teradata connectors.
- Involved in design and development of complex enterprise parallel jobs to extract data into Oracle, Teradata and flat files.
Environment: IBM InfoSphere DataStage 11.3/9.1/8.7/8.1/7.1, XML, Oracle, Teradata, SharePoint, UNIX, HDFS.
Confidential, Bloomington, IL
Java/J2EE Programmer
Responsibilities:
- Developing J2EE web services involving all the stages of Software Development Life Cycle.
- Involve in the requirements gathering, Design, Development, Unit testing and Bug fixing.
- UseDesign Patterns like MVC, Business Delegate, Service Locator, Session Facade and DAO.
- Developed the functionalities using Agile Methodology.
- Involved in writingThread Safe blocks formultithreadaccess to make valid transactions.
- Developed and implemented responsive web pages using JSF Primefaces while maintaining high usability standards.
- Created and injected Spring services, Spring boot, Spring controllers and DAOs to achievedependency injectionand to wire objects of business classes.
- UsedSpring Inheritanceto develop beans from already developed parent beans.
- Develop and Implement interface forSOAP Web Services usingJAX-WSframework involving Global Transactions.
- UsedAJAXextensively to implement front end /user interface features in the application.
- DevelopedWeb Services clientsto consume those Web Services as well otherenterprise wide Web Services.
- UseIBM DB2for creating databases and performing DB2 operations on the tables.
- UsedIBM Pure Queryfor generating JDBC connection beans that can be used for connecting DB2 database.
- Database development required creation of new tables, inserting data into the tables, updating data in the tables and deleting data from the tables and required SQL tuning to reduce the response time in the application.
- Deployed the application on IBM Web Sphere Application Server 8.5.
- Worked closely with QA team and fixed QA bugs as well as production issues with a quick turnaround time.
- UsedSplunkto get the Testing, Pre-productions and Production logs.
- Focused onTest Driven Development; thereby creating detailedJUnittests for every single piece of functionality before writing the functionality.
- UsedApache Mavenfor project management and building the application.
- SVNbeing used for project management and version management.
Environment: J2EE, Java 1.7, Spring framework, Hibernate, JSP 2.0, JSR303, JDBC, AJAX, JAX-WS Web services, SOAP, XML, Java Beans, Apache Axis2, JQuery, JavaScript, AngularJS, IBM DB2, IBM Pure Query
Confidential
Java Developer
Responsibilities:
- Worked on both WebLogic Portal 9.2 for Portal development and WebLogic 8.1 for Data Services Programming.
- Developed the presentation layer using JSP, HTML, CSS and client validations using JavaScript.
- Used GWT to send AJAX request to the server and updating data in the UI dynamically.
- Developed Hibernate 3.0 in Data Access Layer to access and update information in the database.
- Used JDBC, SQL and PL/SQL programming for storing, retrieving, manipulating the data.
- Involved in designing and development of the ecommerce site using JSP, Servlet, EJBs, JavaScript and JDBC.
- Used Eclipse 6.0 as IDE for application development Configured Struts framework to implement MVC design patterns.
- Validated all forms using Struts validation framework and implemented Tiles framework in the presentation layer.
- Designed and developed GUI using JSP, HTML, DHTML and CSS. Worked with JMS for messaging interface.
- Used Hibernate for handling database transactions and persisting objects deployed the entire project on WebLogic application server.
- Used AJAX for interactive user operations and client side validations Used XSL transforms on certain XML data.
- Used XML for ORM mapping relations with the java classes and the database.
- Developed ANT script for compiling and deployment.
- Performed unit testing using Junit.
Environment: Java/J2EE, Oracle 10g, SQL, PL/SQL, JSP, EJB, Struts, Hibernate, WebLogic 8.0, HTML, AJAX, Java Script, JDBC, XML, JMS, XSLT, UML, JUnit, Log4j, Eclipse 6.0.
