- 7+ years of strong experience in Analysis, Design and Development of Business Intelligence Solutions in Data Warehousingusing Talend 6.2 and Informatica PowerCenter 9.5/9.1/8.6/8.1 as an ETL tool on Windows and UNIX based operating systems.
- Application Using ETL tools like Talend 6.3 and IBM InfoSphere Data Stage 9.1.
- Expertise in Integration Services (SSIS) and SQL Server Reporting Services (SSRS) with Good knowledge in SQL Server Analysis Services (SSAS)
- Exposure of ETL methodology for supporting Data Extraction, Transformation and Loading process in a corporate - wide ETL solution using Talend Open Source for Data Integration 6.2.
- Experience in developing ETL mappings, transformations and implementing source and target definitions in Talend.
- 4+ years of experience with Talend Open Studio (6.x/5.x) and 3+ years of experience with Talend Admin Console (TAC)
- Experience in working with Data Warehousing Concepts like OLAP, OLTP, Star Schema, Snow Flake Schema, Logical Data Modeling, Physical Modeling, Dimension Data Modeling
- Converting Large XML files to Multiple XML files as required by downstream application.
- Significant experience with Data Extraction, Transformation and Loading (ETL) from disparate data sources such as multiple relational databases and also worked on integrating data from flat files, CSV files and XML files into a common reporting and analytical data model.
- Extensive experience in Performance Tuning of SQL queries & Views
- Development of automated daily, weekly and monthly system maintenance tasks such as database backup, database integrity verification and index updates
- Experience in using AWS cloud components and connectors to make to make API calls for accessing data from cloud storage (Amazon S3, Redshift) in Talend Open Studio.
- Experience in Service oriented development using Talend ESB.
- Strong understanding of RDBMS concepts and experience in writing PL/SQL and SQL statements in databases.
- Extensively created mappings in Talend using t-Map, t-Join, t-Replicate, t-Parallelize, t-Java, t-Java row, t-Die, t-Aggregate Row, t-Warn, t-Log Catcher, t-Filter, t-Global map etc
- Highly proficient in the integration of various data sources involving multiple relational databases like Oracle, MS SQL Server, Teradata, DB2 and non-relational sources like COBOL Files and Flat File.
- Strong understanding of the principles of DW using fact tables, dimension tables and star/snowflake schema modeling.
- Experience in DI, MDM and ESB.
- Experience of Hadoop Architecture and various components such as HDFS, Name Node, Data Node, Job Tracker, Task Tracker, YARN and Map Reduce.
- Experience on cloud configuration in Amazon web services(AWS).
- Experience with the Architecture SOA, TOGAF and MDM modeling.
- Expertise on Exception Handling Mappings for Data Quality, Data Cleansing and Data Validation.
- Experience in Trouble shooting andimplementing Performance tuning at various levels such as Source, Target, Mapping, Session and System in ETL Process. Have heavily worked on performance tuning long running Informatica mappings using Pushdown optimization and session partitioning.
- Strong Data Analysis and Data Profiling background using Informatica Analyst, Informatica Data Explorer (IDE)and data cleansing background using Informatica Data Quality (IDQ).
- Experience in working with Standardiser, Parser, Match, and Merge & Consolidation Transformations using IDQ.
- Worked with SQL/PL-SQL to write Complex SQL queries, Stored Procedures, Triggers, Functions & PL/SQL packages.
- Experience in Installing and Configuring Talend Administrator Center, Jobserver and Talend Runtime server.
- Create User, Projects, Configure SVN/GIT repositories, Nexus repository and Command Line in TAC.
- Experience on Scheduling Talend Jobs by using third party utilities like Crontab, Windows Scheduler etc.
- Experience on Scheduling Talend Jobs by using Job Conductor and Execution Plan in TAC Server.
- Configuring mail notification in TAC server to send error notification to users .
- Enabled log4j to capture exceptions in Talend Job.
- Experience on creating Audit tables to capture details of the Talend Job like Inserted rows, Updated rows, Job start time, End time etc.
ETL/Middleware Tools: Talend 5.5/5.6/6.2, Informatica Power Center 9.5.1/9.1.1/8.6.1/7.1.1
Business Intelligence Tools: Business Objects 6.0, Cognos 8BI/7.0.s, Sybase, OBIEE 11g/10.1.3.x
RDBMS: Oracle 10g/9i, Netezza, Teradata, MS SQL Server, DB2, MySql, MS Access.
Programming Skills: SQL, Oracle PL/SQL, Unix Shell Scripting, HTML, DHTML, XML, Java, .Net
Tools: TOAD, SQL Plus, SQL*Loader, Quality Assurance, Soap UI, Fish eye, Subversion, Share Point, IP switch user, Teradata SQL Assistant.
Operating Systems: Windows 8/7/XP/NT/2x, Unix-AIX, Sun Solaris 8.0/9.0.
Scheduling Tool: Zena, Maestro, Control M, Informatica Scheduling.
Confidential - Portland, OR
- Participated in all phases of development life-cycle with extensive involvement in the definition and design meetings, functional and technical walkthroughs.
- Created Talend jobs to copy the files from one server to another and utilized Talend FTP components.
- Design, develop, validate and deploy the Talend ETL processes for the DWH team using HADOOP (PIG, HIVE) on Hadoop .
- Used ETL methodologies and best practices to create Talend ETL jobs .
- Followed and enhanced programming and naming standards.
- Created and deployed physical objects including custom tables, custom views, stored procedures, and Indexes to SQL Server for Staging and Data-Mart environment.
- Design and Implemented ETL for data load from heterogeneous Sources to SQL Server and Oracle as target databases and for Fact and Slowly Changing Dimensions SCD-Type1 and SCD-Type2.
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce, Spark and loaded data into HDFS.
- Implemented installation and configuration of multi-node cluster on cloud using AWS on EC2.
- Utilized Big Data components like tHDFSInput, tHDFSOutput, tPigLoad, tPigFilterRow, tPigFilterColumn, tPigStoreResult, tHiveLoad, tHiveInput, tHbaseInput, tHbaseOutput, tSqoopImport and tSqoopExport .
- Extensively used tMap component which does lookup & Joiner Functions, tjava, tOracle, txml, tdelimtedfiles, tlogrow, tlogback components etc. in many of my Jobs Created and worked on over 100+components to use in my jobs.
- Used Talend most used components ( tMap, tDie, tConvertType, tFlowMeter, tLogCatcher, tRowGenerator, tSetGlobalVar, tHashInput&tHashOutput and many more).
- Developed integration services using SOA, Mule ESB, Web Services, SOAP, and WSDL.
- Copy data to AWS S3 for storage and use COPY command to transfer data to Redshift. Used Talend connectors integrated to Redshift.
- Established SOA Center of Excellency, Technical and SOA standards, Governance, Service Identification Framework and Life cycle Management guide lines rendering the Business Strategic requirements into technical specifications.
- Developed jobs to expose HDFS files to Hive tables and Views depending up on the schema versions.
- Have used AWS components (Amazon Web Services) - Downloading and uploading data files (with ETL) to AWS system using S3 talend components.
- Used AWS EC2 instances to store and schedule talend jobs.
- Created many complex ETL jobs for data exchange from and to Database Server and various other systems including RDBMS, XML, CSV, and Flat file structures .
- Created Implicit, local and global Context variables in the job.
- Imported data from RDBMS (MySQL, Oracle) to HDFS and vice versa using Sqoop (Big Data ETL tool) for Business Intelligence, visualization and report generation.
- Worked on Talend Administration Console (TAC) for scheduling jobs and adding users.
- Worked on various Talend components such as tMap, tFilterRow, tAggregateRow, tFileExist, tFileCopy, tFileList, tDieetc.
- Developed stored procedure to automate the testing process to ease QA efforts and also reduced the test timelines for data comparison on tables.
- Automated SFTP process by exchanging SSH keys between UNIX servers.
- Worked Extensively on Talend Admin Console and Schedule Jobs in Job Conductor .
- Involved in production and deployment activities, creation of the deployment guide for migration of the code to production, also prepared production run books.
Environment: Talend Data Integration 6.1/5.5.1, Talend Enterprise Big Data Edition 5.5.1, Talend Administrator Console, Oracle 11g, Hive, HDFS, Sqoop, Netezza, SQL Navigator, Toad, Control M, Putty, Winscp.
Confidential - Derry, NH
- Interacted with business team to understand business needs and to gather requirements.
- Designed target tables as per the requirement from the reporting team and also designed Extraction, Transformation and Loading (ETL) using Talend .
- Created Technical Design Document from Source to stage and Stage to target mapping.
- Worked with Talend Studio (Development area) & Admin Console (Admin area).
- Created Java Routines, Reusable transformations, Joblets using Talend as an ETL Tool.
- Created Complex Jobs and used transformations like tMap, tOracle (Components), tLogCatcher, tStatCatcher, tFlowmeterCatcher , File Delimited components and Error handling components(tWarn, tDie).
- Identified performance issues in existing sources, targets and Jobs by analyzing the data flow, evaluating transformations and tuned accordingly for better performance.
- Manage all technical aspects of the ETL Jobs process with other team members.
- Worked with Parallel connectors for parallel processing to improve job performance while working with bulk data sources.
- Worked on generating the web services classes by using Service Oriented Architecture SOA, WSDL, UDDI, and SOAP.
- Developed mappings to load Fact and Dimension tables, SCD Type 1 and SCD Type 2 dimensions and Incremental loading.
- Created contexts to use the values throughout the process to pass from parent child to child jobs and child to parent jobs.
- Worked on Joblets(reusable code) & Java routines in Talend.
- Expertise in Service Oriented Architecture (SOA) and Involved in publishing Web Services that include components like WSDL, SOAP, UDDI.
- Performed Unit testing and created Unix Shell Scripts and provided on call support.
- Schedule Talend Jobs using Job Conductor (Scheduling Tool in Talend ) - available in TAC.
- Retrieved data from Oracle and loaded into SQL Server data Warehouse .
- Created many complex ETL jobs for data exchange and to Database Server and various other systems including RDBMS, XML, CSV , and Flat file structure.
- Created and reviewed scripts to create new tables, views, queries for new enhancement in the applications using TOAD.
- Monitoring the Data Quality, Generating weekly/monthly/yearly statistics reports on production processes - success / failure rates for causal analysis as maintenance part andEnhancing existing production ETL process.
- Development of high level data dictionary of ETL data mappings and transformations from a series of complex Talend data integration jobs.
Environment: Talend Platform for Big Data 5.5.1/5.6.2 , UNIX, Oracle 10g, Oracle, TAC.
Confidential - Birmingham, AL
- Design, Develop, Test, Implement and Support of Data Warehousing ETL using Informatica.Used JIRA to create, implement and deploy ETL related stories.Participated in daily scrum and bi weekly iteration planning as part of agile environment.
- Research, analyze and prepare logical and physical data models for new applications and optimize the data structures to enhance data load times and end-user data access response times.Used SQL tools like TOAD to run SQL queries and validated the data. Created Java programs to read data from web services and load into Teradata.
- Created Unit test environments for ETL processes using sql scripts, JUNIT, HSQLDB (In memory database) and Liquibase.
- Developed stored procedures/views in Teradata and used in Informatica for loading scoring tables.
- Performed ETL using different sources like MYSQL tables, CSV, fixed length files and loaded into Teradata, HDFS and hive targets.
- Manage and scheduled ETL jobs using Informatica Workflow manager in development and production environment.
- Prepare high level design document, detail design document, system requirement document, technical specifications, table level specs and test plan documents.
- Extract data from legacy systems to staging area and then cleanse, homogenize, process and load into the data warehouse.
- Develop MERGE scripts to UPSERT data into Teradata for an ETL source.Worked on writing pig/hive/hcatalog scripts to process huge data files like web clicks data.
- Used GIT as version control for the code and implemented branching for different environments and provided 24x7 production support for the ETL processes.
Environment: Informatica 8.6/IDQ, JIRA, Java, Maven, GIT, FLUX, Teradata, MySQL, Putty, XML, JUNIT, Hadoop, Apache Pig, Hive, Web Services, OBIEE, Microsoft Office, Oracle 10g/11g.
SQL Server Developer
- Wrote complex stored procedures, User Defined Functions, Triggers using T-SQL.
- Used SSIS jobs for importing data from the flat files that brings the data to the application tables.
- Created SSIS packages to extract data from OLTP and transformed to OLAP systems and Scheduled Jobs to call the packages and Stored Procedures.
- Created DTS packages for data transfer between the two environments.
- Security issues related to logins, database users, and application roles and linked servers.
- Performance tuning of SQL queries and stored procedures using SQL profiler and Index tuning advisor.
- Administered of all SQL Server database objects, logins, users and permissions in each registered server.
- Involved in the analysis and profiling of source data, creating prototypes and sample reports to help with requirements and design (SSRS)
- Developed DTS packages to copy tables, schemas and views and to extract data from Excel and Oracle using SSIS
- Resolved any deadlocks issues with Databases/Servers on a real-time basis.
- Wrote scripts for generating Daily Backup Report, verifying completion of all routine backups, log space utilization monitoring etc.
- Generated reports in SQL Server Reporting Services (SSRS) using table control, matrix control for crosstab reports.
- Developed Tabular, Drilldown, Parameterized, Charts, Sub-Reports using SSRS.
- Defined Report Layouts for the report design as per the need.
- Installed SSRS, Configured the Report Server and Deployed Reports.
Environment: MS SQL Server Management Studio 2008/2005, Query Analyzer, SQL Profiler, Index Tuning Wizard, MS SQL Server Integration Services (SSIS), MSSQL Server Analysis Services (SSAS), MS SQL Server Reporting Services (SSRS), Microsoft Visual source Safe