- Around 8+ years IT experience in Design, Development, Maintenance, Enhancements and Production Support of which includes Data warehousing and Legacy applications using ETL tools like Talend /Informatica/SAS.
- Experience in all aspects of Analysis, Design, Testing, Development, Implementation and Support of Relational Database (OLTP), Data Warehousing Systems (OLAP) and Data Marts in various domains.
- Experience in working with Data Warehousing Concepts like OLAP, OLTP, Star Schema, Snow Flake Schema, Logical Data Modelling, Physical Modelling and Dimension Data Modelling.
- Utilized tStat Catcher, tDie, tLog Row to create a generic job to store processing stats.
- Involved in extracting user's data from various data sources into Hadoop Distributed File Systems (HDFS)
- Experience in adhering software methodologies like waterfall, Agile.
- Excellent understanding and knowledge of NOSQL databases.
- Experienced in creating Generic schemas and creating Context Groups and Variables to run jobs against different environments like Dev, Test and Prod.
- Thorough knowledge of addressing Performance Issues and Involved in query tuning, Index tuning, Data profiling and other database related activities.
- Extensively created mappings in Talend using tMap, tJoin, tReplicate, tParallelize, tJava, tJava row, tDie, tAggregate Row, tWarn, tLog Catcher, tFilter, tGlobal map etc.
- Wrote Hive and Pig queries for data analysis to meet the business requirements.
- Experienced in scheduling Talend jobs using Talend Administration Console (TAC)
- Experience with Talend DI Installation, Administration and development for data warehouse and application integration.
- Tracking Daily data load, Monthly data extracts and send to client for their verification.
- Strong experience in designing and developing Business Intelligence solutions in Data Warehousing using ETL Tools.
- Excellent understanding and best practice of Data Warehousing Concepts, involved in Full Development life cycle of Data Warehousing.
- Experienced in working with different data sources like Flat files, Spreadsheet files, log files and Databases.
- Hands - on experience across all stages of Software Development Life Cycle (SDLC) including business requirement analysis, data mapping, build, unit testing, systems integration and user acceptance testing.
- Excellent interpersonal and communication skills, and is experienced in working with senior level managers, business people and developers across multiple discipline
Databases: Teradata,Oracle12c/11x/10g, DB2, Microsoft SQL Server, Hive, Sybase,Redshift DB
Programming: T-SQL, PL/SQL, HTML, XML.
Environment: Windows, UNIX & Linux
Languages: SQL, Java, .Net, BASE SAS.
Other Tools: SQL Navigator, Putty, MS-Office.
Talend-Big Data Integration Developer
Confidential, Chicago, IL
- Responsible to build the jobs by looking to the ETL Specification documents.Worked closely with Business Analysts to review the business specifications of the project and to gather the ETL requirements.
- Closely worked with Data Architects in designing of tables and even involved in modifying technical Specifications.
- Involved in Extraction, Transformation and Loading of data.
- Experienced in Creating the Context load using the parameter variables.
- Worked on Batch Jobs and scheduling,
- Used Components in Talend Like (tMap, Tfilelist, Tjava, Tlogrow, ToracleInput, ToracleOutput, tsendEmail etc)
- Extensively Used components like tMap, tDie, tConvertType, tFlowMeter, tLogCatcher, tRowGenerator tfileList etc.
- Utilized Big Data components like tHDFSInput, tHDFSOutput, tHiveLoad, tHiveInput, tHbaseInput, tHbaseOutput, tSqoopImport, tSqoopExport, tHiveOutput, tHiveRow, tHiveConnection.
- Data ingestion with different data sources and load into redshift.
- Developed jobs to send and read data from AWS S3 buckets using components like tS3Connection, tS3BucketExist, tS3Get, tS3Put.
- Experienced in implementing the project end to end Solutions.
- Involved in Data Extraction from Flat files and XML files using Talend and Java as Backend Language.
- Used Talend to load the data into our warehouse systems
- Used debugger and breakpoints to view transformations output and debug mappings.
- Develop ETL mappings for various Sources (.TXT, .CSV, XML) and load the data from these sources into relational tables with Talend Enterprise Edition.
- Worked on Global Context variables, Context variables, and extensively used over 70+components in Talend to create jobs.
- Worked on Error handling techniques and tuning the ETL flow for better performance.
- Worked Extensively TAC (Admin Console), where we Schedule Jobs in Job Conductor.
- Migrated the code and release documents from DEV to QA (UAT) and to Production.
- Design and Implemented ETL for data load from heterogeneous Sources to SQL Server and Oracle as target databases and for Fact and Slowly Changing Dimensions SCD-Type1 and SCD-Type-2.
Environment: Talend Enterprise Big Data Edition 6.5, Talend Administrator Console, MS SQL Server, Oracle 11g, HDFS, Hive, Amazon Redshift.
Confidential, DesMoines, IA
- Participated in all phases of development life-cycle with extensive involvement in the definition and design meetings, functional and technical walkthroughs.
- Created Talend jobs to copy the files from one server to another and utilized Talend FTP components
- Created and managed Source to Target mapping documents for all Facts and Dimension tables
- Used ETL methodologies and best practices to create Talend ETL jobs.
- Followed and enhanced programming and naming standards.
- Created and deployed physical objects including custom tables, custom views, stored procedures, and Indexes to SQL Server for Staging and Data-Mart environment.
- Extensively used tMap component which does lookup & Joiner Functions, tjava, tOracle, txml, tdelimtedfiles, tlogrow components etc. in many of my Jobs Created and worked on over 100+components to use in my jobs.
- Used Talend most used components (tMap, tDie, tConvertType, tLogCatcher, tRowGenerator, tHashInput & tHashOutput and many more).
- Created many complex ETL jobs for data exchange from and to Database Server and various other systems including RDBMS, XML, CSV, and Flat file structures.
- Created Implicit, local and global Context variables in the job.
- Worked on Talend Administration Console (TAC) for scheduling jobs and adding users.
- Worked on various Talend components such as tMap, tFilterRow, tAggregateRow, tFileExist, tFileCopy, tFileList, tDie etc.
- Involved in production n deployment activities, creation of the deployment guide for migration of the code to production, also prepared production run books.
Environment: Talend Data Integration 6.1/5.5.1,, Talend Administrator Console, Oracle 11g, Hive, HDFS, SQL Navigator, Toad, Putty, Winscp.
SQL/ETL Inforamtica Developer
- Coordinated with Business Users for requirement gathering, business analysis to understand the business requirement and to prepare Technical Specification documents (TSD) to code ETL Mappings for new requirement changes.
- Analyze and create low level design document (LLD) and mapping document.
- Performed analysis, design, development, Testing and deployment for Ingestion, Integration, provisioning using Agile Methodology.
- Created jobs to perform record count validation and schema validation.
- Developed jobs that are used for ETL purposes.
- Administer all requests and analyze issues and provide efficient resolution for same.
- Design all program specifications and perform required tests in same.
- Design all programs and systems and associate documentation for same.
- Prepare all program and system implementation for all informatics programs.
- Monitor all production issues and inquiries and provide efficient resolution for same.
- Performed unit testing and integration testing after the development and got the code reviewed.
- Designed and Developed Oracle PL/SQL and UNIX Shell Scripts, Data Import/Export.
- Identified and fixed the Bottle Necks and tuned the Mappings and Sessions for improving perform.
- Tuned both ETL process as well as Databases.
Environment: Talend Studio 5.2.2 /5.6, Talend Studio Big Data Platform, HBase, XML files, Flat files, HL7 files, HDFS, Hive, Oracle 11g, Business Objects, UNIX, WinSCP, Clear Case, Clear Quest, Erwin, PL/SQL, Toad, Windows 7 Pro, TFS, JIRA.
SAS Developer/SQL/ETL Informatica
- Plan Design and implement application database code objects, such as stored procedure and views.
- Worked on different ETL tools like Informatica, BASE SAS fot the data movement.
- Build and Maintain SQL scripts, indexes and complex queries for data analysis and extraction.
- Created complex ETL jobs for data exchange from and to Database Server and various other systems including RDBMS, XML, CSV, and Flat file structures.
- Created contexts to use the values throughout the process to pass from parent to child jobs and child to parent jobs.
- Developed joblets that are reused in different processes in the flow.
- Created, modified/Updates SAS code to in corporate changes in new releases of the product.
- Produced ad hoc reports in SAS using .txt files upon request for in-depth analysis.
- Data manipulation by merging, appending and sorting datasets.
- Extensive use of PROC SQL to create subsets of data for further analysis.
- Extensive use of DATA STEP, PROC MEANS, PROC SQL, MACROS, PROC FREQ, PROC TABULATE, DATA NULL, PROC MEANS, PROC SUMMARY, RETAIN.
- Generated output files in various formats like text, MS Excel.
- Worked on big project like large group Migration, where the data has to be moved from Staging data base to the Target Production data.
- Validated the data and worked on large number of defect fixes to satisfy the business for the required exact formats for the data.
- Figured out the problem to fix the defects, if the data is not reflected in the target as expected like Code changes, timing issues (when running the Cycles) etc.
- Worked on CDR (Change Data Requests), Installation Documents, and attended the meetings to promote the code to the production (live).
Environment: XML files, Flat files, HL7 files, HDFS, Hive, Oracle 11g, Business Objects, UNIX, WinSCP, Clear Case, Clear Quest, Erwin, PL/SQL, Toad, Windows 7 Pro, TFS, JIRA.