Etl Developer / Architect Resume
NJ
PROFESSIONAL SUMMARY:
- 12 years of experience in ETL DataStage, Talend, Hadoop framework, Informatica, Data Analysis, Data Warehouse Implementation, Data migration and Data Conversion
- Extensive experience in Talend Integration Suit 6.2/6.1/5.6/5.4 and Data Stage 11.3/9.1/8.7/8.5/8.0.1/7.5. x using Components like Data Stage Designer, Data Stage Manager, Data Stage Director, Data Stage Administrator, Information server and Parallel Extender.
- Sound experience in Unix Shell scripting.
- Sound experience in IBM DataStage.
- Sound experience in UNIX Shell scripting.
- Experience on Teradata, SQL Server and Netezza databases.
- Experience on AWS Redshift.
- RDBMS experience with Oracle 10g/9i/8i, PL/SQL.
- Implementation of ETL functionality of Data warehouse and Data migration applications.
- Extensive experience in loading high volume data, and performance tuning.
- Strong background in MPP data warehouse environments, architecture and its implementation.
- Strong experience in Data conversion from legacy source systems to new target system.
- Extensive experience in end - to-end implementation of Business Intelligence (BI) projects, especially in big data space, Business Intelligence and Analytics developments.
- Working experience on Hadoop framework, UNIX scripts, Pig, Hive.
- Working experience on SQL server platform.
- Working experience in SSIS and SSRS.
- Analyzed the Source Data and designed the source system documentation.
- Strong experience in working with Business Analyst to understand business requirements from business owners in order to transform business requirements into effective technology solutions.
- Created the data mappings and conversion logic to cleanse the data.
- Strong experience on writing the SQL queries.
- Experience on Scheduling tools AUTOSYS, CONTROL M, Tidal
- Strong understanding of tuning and optimization of jobs.
- Experience in handling bulk volume of data to cleanse and load into target.
- Good knowledge in data warehouse concepts like Star Schema, Snow Flake, Dimension and Fact tables.
- Excellent data analysis skills.
- Experience in Converting the Business Logic into Technical Specifications.
TECHNICAL SKILLS:
ETL Tools: Talend 6.2/6.1/5.6, DataStage 11.5/ 11.3/9.1/8.7/8.5/8.0.1/7.5. x, Informatica 7x/8xBig Data Technologies Hadoop ecosystem, HDFS, Map Reduce, Hive, PIG, Sqoop, NOSQL
Reporting Tools: Business Objects XI R2, Cognos8.x/7.x, Micro strategy and MS Access Reports.
Operating System: Windows NT/XP, UNIX. Sun Solaris 8/7.0, IBM AIX 4.3
Database: Oracle 9i/10g, 11g, SQL Server 2008/2012, DB2, Teradata, Netezza, AWS Redshift
Language: C, Java, SQL, PL/SQL, UNIX
Modelling Tools: Visio, Erwin 7.1.2
PROFESSIONAL EXPERIENCE:
Confidential, NJ
ETL Developer / Architect
Responsibilities:
- Used ETL methodologies and best practices to create Talend ETL jobs
- Extracted data from JSON files applied business logic to load them in the staging database as well as flat files.
- Automating ETL jobs using Talend components.
- Extracted data from JSON files.
- Loading data from delta files into S3 and AWS Redshift.
- Used Talend components for loading data in AWS Redshift.
- Designed ETL Jobs/Packages using Talend Integration Suite (TIS).
- Responsible for all technical and system specifications documents for all ETL processes.
- Implement procedures for maintenance, monitoring, backup, and recovery operations for the ETL environment.
- Involved in Talend administration activities.
- Involved in deployment of Talend jobs in Test, Prod environment.
Environment: Talend, AWS Redshift, Winscp, Putty, Unix
Confidential, MN
ETL Developer / Analyst
Responsibilities:
- Used ETL methodologies and best practices to create Talend ETL jobs
- Extracted data from flat files/ databases applied business logic to load them in the staging database as well as flat files.
- Converted Datastage jobs into Talend jobs.
- Designed ETL Jobs/Packages using Talend Integration Suite (TIS).
- Responsible for all technical and system specifications documents for all ETL processes.
- Implement procedures for maintenance, monitoring, backup, and recovery operations for the ETL environment.
- Creating the configuration file for maintaining Log files.
- Worked in log4j configuration.
- Extracted Files from Data Lake Hadoop environment.
- Loading Data into Hive tables.
- Worked on pig scripts.
- Extracted data from flat files/ databases applied business logic to load them into Netezza tables.
- Used Netezza utilities Nzload and External Tables to load in Netezza tables in Talend.
- Designed ETL Jobs/Packages using Talend Integration Suite (TIS).
- Created Talend Mappings to populate the data into dimensions and fact tables
- Implemented new users, projects, tasks within multiple different environments of TAC (Dev,Test,Prod)
- Worked with clients and created specific MRS documents to ensure mapping was to their specification.
- Worked with third party vendors on specific requirements needed for new feeds.
- Created and managed Source to Target mapping documents for required jobs.
- Reporting (daily, weekly, and monthly) on health and performance of the ETL environment and jobs.
Environment: Talend Integration Services 5.6, Datastage, Netezza, Hadoop, DB, SVN, Talend Administrative Console (TAC)
Confidential, GA
ETL Developer / Analyst
Responsibilities:
- Designed ETL Jobs/Packages using Talend Integration Suite (TIS)
- Designed ETL Jobs using DataStage.
- Converted DataStage jobs into Talend jobs.
- Worked on different data sources such as Teradata, Flat files etc.
- Worked on Teradata utilities like Fast Load, multiload, Bteq scripts.
- Worked in Big Data components PIG and HIVE and HDFS
- Used tLogCatcher, tStatsCatcher, tDie, tLogRow to main Log information.
- Worked with third party vendors on specific requirements needed for new feeds.
- Created and managed Source to Target mapping documents for required jobs.
- Ensured jobs, projects and MRS documents were up to date on the SVN within different environments.
- Was involved in the creation and fixing of jobs for four different teams. One of which was the release team for Sony.
- Worked on heterogeneous data sources and targets such as Teradata.
- Created physical database objects from data model to physical database like tables, indexes, views, and triggers. Also created DDL files to be deployed by production DBAs.
Environment: Talend Integration Services (TIS 5.6), DataStage, Hadoop, Teradata, Winscp, Hadoop
Confidential, Providence, RI
Sr. ETL Data Stage Developer/Architect
Responsibilities:
- Performed Column Analysis, Primary Key Analysis, Foreign Key Analysis and Cross Domain Analysis using IBM Information Analyzer.
- Analyze and interpret all complex data on all target systems and analyze and provide resolutions to all data issues and coordinate with data analyst to validate all requirements.
- Monitor all business requirements and validate all designs and schedule all ETL processes and prepare documents for all data flow diagrams.
- Developed UNIX shell script to develop some secure format.
- Provided technical expertise and leadership to develop integration architecture solutions.
- Developed the design documents.
- Designed and developed jobs using Data Stage/Quality Stage 11.3
- Involved in tuning Data Stage jobs and sequencers.
- Provide solution for the Customer Service Requests and ad-hoc cases, requiring application related attention.
- Developed parallel jobs using stages such as Join, Merge, Lookup, Remove Duplicates, Copy, Filter, Funnel, Dataset, Lookup, Pivot, and Sort, Surrogate key Generator, Change Data Capture (CDC), Modify
- Developed UNIX shell scripts and run the jobs through from UNIX shell scripts
- Provide support to all ETL schedule and maintain compliance to same and develop and maintain various standards to perform ETL codes and maintain an effective project life cycle on all ETL processes.
- Perform root cause analysis on all processes and resolve all production issues and validate all data and perform routine tests on databases and provide support to all ETL applications.
- Scheduled jobs using AUTOSYS
- Worked on agile methodology to develop, test and deploy ETL.
Environment: Data Stage 11.3, Winscp, Putty, Oracle 11g, Teradata, Autosys
Confidential
Sr. ETL Developer/Architect
Responsibilities:
- Responsible for all technical and system specifications documents for all ETL processes and perform unit tests on all processes and prepare required programs and scripts.
- Analyze and interpret all complex data on all target systems and analyze and provide resolutions to all data issues and coordinate with data analyst to validate all requirements.
- Designed parallel jobs using stages such as Join, Merge, Lookup, Remove Duplicates, Copy, Filter, Funnel, Dataset, Lookup, Pivot, and Sort, Surrogate key Generator, Change Data Capture (CDC), Modify.
- Developed UNIX shell scripts and run the jobs through from UNIX shell scripts
- Developed UNIX shell scripts.
- Provide support to all ETL schedule and maintain compliance to same and develop and maintain various standards to perform ETL codes and maintain an effective project life cycle on all ETL processes.
- Monitor all business requirements and validate all designs and schedule all ETL processes and prepare documents for all data flow diagrams.
- Participated in Hadoop framework.
- Scheduled jobs using AUTOSYS scheduler utility based on the requirements and monitored the production processes closely for any possible errors
- Worked on agile methodology to develop, test and deploy ETL.
Environment: Data Stage 9.1, Winscp, Putty, Teradata, Rally, Teradata, SQL Server, UNIX
Confidential, Seattle, WA
Sr. Data Stage Developer
Responsibilities:
- Participated in requirements session along with the business analyst to understand information and analytics requirement from the business
- Work closely with Architects and Developers to develop jobs in such a way it can be re used for different regions.
- Extensively used Data Stage for extracting, transforming and loading databases from sources including Oracle, DB2 and Flat files.
- Co-ordinating and allocating the team for Development of jobs.
- Involved in performance tuning of Jobs and Data Analysis to resolve defects with migrated data
- Extensively worked with Job sequences using Job Activity, Email Notification, Sequencer, Wait for File activities to control and execute the Data Stage Parallel jobs.
- Worked on UNIX scripts.
- Performed the Unit testing for jobs developed to ensure that it meets the requirements.
- Created Unit test cases and Unit test results.
- Scheduled the jobs using Autosys.
- Provided technical assistance and support to IT analysts and business community.
Environment: Data Stage 8.5/9.1, Winscp, Information Analyzer, Putty, Autosys, Oracle, UNIX
Confidential, Kansas City, MO
Sr. Data Stage Developer
Responsibilities:
- Participated in designing, developing and implementing solutions based on detailed requirements.
- Participated in ETL Design process and development process.
- Worked with Business Analyst to understand business requirements from business owners in order to transform business requirements into effective technology solutions.
- Applications worked on Ariba.
- Data Analysis to resolve defects with migrated data.
- Work closely with Senior Architects and Developers to develop standards as well as working with Jr. ETL developers to ensure the development of re-usable objects.
- Involved in Analysis, Development, Testing and Implementation of ETL jobs and loading the databases from Oracle 9i to 11g with multiple source systems.
- Development of ETL jobs to migrate bulk volume of data.
- Used Administrator to administer the locks on the jobs and other Administration activities for data stage server.
- Involved in performance tuning of Jobs.
- Extensively worked with Job sequences using Job Activity, Email Notification, Sequencer, Wait for File activities to control and execute the Data Stage Parallel jobs.
- Regularly did the Data Quality Checks on the source system.
- Worked closely with Development and QA to clarify the requirement issues.
- Scheduled the jobs using Autosys.
- Have addressed production, UAT issues, proper action was taken accordingly based on priority and requirement.
Environment: Data Stage 8.5, Winscp, Putty, Oracle, Araba, UNIX, Autosys
