- Around 6 years of IT Experience on Talend & Datastage experience in ETL Data Warehousing and Business Intelligence applications design, development and Maintenance of Integration Services using Talend, Datastage and Informatica by using RDBMS and NoSQL Databases.
- Experience in Software Development Life Cycle - SDLC (Analysis, Design, Development and Testing), requirement gathering, client interaction, Use Case Design and understanding.
- Experience in UNIX scripting, troubleshooting and file handling on the UNIX system. Experience in Data Integration, EDW and Data Mart projects.
- Worked with different components (tMap, tDie, tConvertType, tFlowMeter, tLogCatcher, tRowGenerator, tSetGlobalVar, tHashInput & tHashOutput and many more).
- Worked with different tHDFSInput, tHDFSOutput, tPigLoad, tPigFilterRow, tPigFilterColumn, tPigStoreResult, tHiveLoad, tHiveInput, tHbaseInput, tHbaseOutput, tSqoopImport and tSqoopExport.
- Experience in UNIX file/dataset management to keep the load ready data for all financial transactional data.
- Experience on Water fall and Agile methodology project implementation.
- Involved in complete Software Delivery Life Cycle (SDLC) in large data warehouse environment for financial data system.
- Thorough knowledge in data warehousing, dimensional modeling, data integration, data virtualization, data synchronization, star schema, snow flake schema, ETL development & performance tuning, BI data analysis, SAP integrating, DFS & HDFS cluster segregation.
- Experience in working closely with mainframe applications for the ETL interactions.
- Prepared High-Level design document, Low Level design document, Technical design document for various projects and good at Bug fixing, Code reviews, and Unit & System testing.
- Expertise in creating Data Mapping for various projects and applications. Experience in working in a Multi-Dimensional Warehouse projects.
- Hands on Experience in working with BOXI for report generation.
- Worked on Agile methodology.
Confidential, Jacksonville, FL
Sr. Talend Developer
- Developed complex ETL mappings for Stage, Dimensions, Facts and Data marts load. Involved in Data Extraction for various Databases & Files using Talend
- Created Talend jobs using the dynamic schema feature. Have used Big Data components (Hive components) for extracting data from hive sources.
- Performance tuning - Using the tmap cache properties, Multi-threading and tParallelize components for better performance in case of huge source data. Tuning the SQL source queries to restrict unwanted data in ETL process.
- ELT components - Pushdown optimization technique. Moving the transformation logic to the database side instead of handling at Talend side. Database tables are indexed properly, and data is huge then ELT method can provide to be much better option in terms of performance of the Job
- Have used AWS components (Amazon Web Services) - Downloading and uploading data files (with ETL) to AWS system using S3 components.
- Used more components in Talend and Few to be mentioned: tjava, toracle, txmlMap, tdelimited files, tlogrow, tlogback components etc. in many of my Jobs Design
- Worked on Joblets (reusable code) & Java routines in Talend
- Experience in using Repository Manager for Migration of Source code from Lower to higher environments.
- Created Projects in TAC and Assign appropriate roles to Developers and integrated SVN(Subversion)
- Used to be On call Support if the Project is deployed to further Phases
- Used Talend Admin Console Job conductor to schedule ETL Jobs on daily, weekly, monthly and yearly basis (Cron Trigger)
Environment: Talend Open studio V(6.1.1)Enterprise Platform for Data management (V6.1.1,5.5.1, 5.6.1), UNIX, Oracle, Microsoft SQL Server management Studio.
Confidential, Cincinnati, OH
- Design and developed end-to-end ETL process from various source systems to Staging area, from staging to Data Marts.
- Developed high level and detailed level technical and functional documents consisting of detailed design documentation function test specification with use cases and unit test documents
- Developed jobs in Talend Enterprise edition from stage to source, intermediate, conversion and target.
- Developed PL/SQL triggers and master tables for automatic creation of primary keys.
- Involved in Talend Data Integration, Talend Platform Setup on Windows and UNIX systems.
- Created job lets in Talend for the processes which can be used in most of the jobs in a project like to Start job and Commit job.
- Created complex mappings in Talend using tHash, tDenormalize, tMap, tJoin, tReplicate, tParallelize, tJava, tjavarow, tUniqueRow, tPivotToColumnsDelimited as well as custom component such as tUnpivotRow.
- Used tStatsCatcher, tDie, tLogRow, tDie, tWarn, tLogCatcher, to create a generic job let to store processing stats into a Database table to record job history.
- Created Talend Mappings to populate the data into dimensions and fact tables.
- Developed complex Talend ETL jobs to migrate the data from flat files to database.
- Implemented custom error handling in Talend jobs and also worked on different methods of logging.
- Prepared ETL mapping Documents for every mapping and Data Migration document for smooth transfer of project from development to testing environment and then to production environment.
- Developed error logging module to capture both system errors and logical errors that contains Email notification and also moving files to error directories.
- Created Talend ETL job to receive attachment files from pop e-mail using tPop, tFileList, tFileInputMail and then loaded data from attachments into database and archived the files
- Created jobs and job variable files for Teradata TPT and load using tBuild command from command line.
- Implemented agile development methodology using XP, Scrum and Kanban/Continuous Flow.
- Created FTP scripts and Conversion scripts to convert data into flat files to be used for Talend jobs.
Environment: Talend 6.0.1/5.5, Oracle 11g, Teradata V 13.0, Teradata SQL Assistant, MS SQL Server 2012/2008, DB2, TOAD, Erwin, AIX, Shell Scripts.
Confidential, Chevy Chase, MD
- Identified source systems, their connectivity, related tables and fields and ensured data suitability for mapping
- Prepared FastTrack Mapping Specifications created metadata layouts in Metadata Workbench and update Business Glossary.
- Prepared Data Mapping Document and Design the ETL jobs based on the DMD with required Tables in the Dev Environment.
- Active participation in decision making and QA meetings and regularly interacted with the Business Analysts &development team to gain a better understanding of the Business Process, Requirements& Design.
- Studied the PL/SQL code developed to relate the source and target mappings. Used DataStage as an ETL tool to extract data from sources systems, loaded the data into the IBM DB2 database.
- Used Data Stage Director and its run-time engine for job monitoring, testing and debugging its components, and monitoring the resulting executable versions on ad hoc or scheduled basis. Also worked for different enhancements in FACT tables.
- Performed Performance testing with different sets of node configuration, different queue and different volumes. Prepared the DML’s for maintenance tables, review, and test and execute them.
- Used Tortoise SVN version control tool for version controlling and movement of code to upper environments like SIT, UAT, Pre-production and Production.
Environment: DataStage 8.1, 8.5 Designer, Director, Teradata V2R6, Tools & Utilities (BTEQ, Fast Export, Multi Load, Fast load, TPUMP), PL/SQL, Oracle 10g, Windows 2000/NT, Unix, Control-M Scheduling.
Confidential, Monterey Park, CA
- Going through the business requirement, preparing the impact analysis document. Prepared technical specification document, upon review of the solution developed the solution using DataStage jobs and sequencers.
- Used sequential file stage as the source for most of the source systems. Developed a file check process that checks the format, volume and date in the file decides whether the right file is being sent by the source and whether the right file is being loaded into the database.
- Created DDL statements for new tables, changes to table structure, index changes, and creation of triggers and stored procedures. Prepared unit test cases and test plans.
- Executed the test cases, captured the results. Supported the SIT testing, UAT testing. Worked on packaging the code with the help of tortoise SVN version controlling tool and worked with respective teams to deploy the code.
- Supported the system post production and worked in co-ordination with the production support teams to resolve any issues.
Environment: DataStage 8.1, Oracle 10g, SQL Programmer, Zena Scheduling, Autosys, Business Objects, TOAD 9.1.