Sr. Etl Developer Resume
Northbrook, IL
SUMMARY
- 8+ years of Database and Data Warehouse development experience covering all phases of SDLC including Requirements Gathering, System Analysis, Design, Development, Testing and Final Release using Agile Methodology.
- Good knowledge on Data Warehouse Architecture, Data Marts, Operational Data Store (ODS), OLAP, OLTP, Data Modeling like Dimensional Data Modeling, Star Schema Modeling, Snowflake Modeling, FACT and Dimensions Tables using Analysis Services.
- Interpreting and working with data models, data flows and data dictionaries, creating source to target mappings, logical & physical data modeling using EA Erwin Diagrams and Visio Diagrams.
- Hands on Experience working with various databases: Oracle11g & 12c, SQL Server 2008, Netezza, Snowflake & Redshift.
- Proficient in writing SQL Queries, creating stored procedures, Indexes, triggers and views.
- Strong Experience in (ETL) Extract Transform & Load large DWH and Data Marts using DataStage, Talend ETL tools and further generating reports using Tableau Reporting tool.
- Hand on Experience Analyzing BI Needs, implementing dash boarding, data visualizations, and analytics using tableau desktop and server.
- Extract and integrate data from various data sources like Legacy Systems, databases, sequential files, XML and Flat files.
- Extensively worked on DataStage Parallel and Sequence Jobs using Various Stages like Join, Lookup, Merge, Transformer, Filter, Funnel, Sort, Aggregator, CDC, Remove Duplicate etc.
- Experience in identifying dependencies between jobs and scheduling Sequence and parallel jobs using DataStage Director, UNIX scripts and scheduling tools: Autosys and Control - M.
- Extensively created mappings in Talend DI and Big Data using various components available in Palette.
- Good knowledge on Talend Administration Center (TAC), implementing time & event-based schedule, performance tuning and troubleshooting.
- Experience in creating Joblets and context group/variables in Talend for the processes which can be reused in most of the jobs in a project.
- Proficient in creating sub jobs in parallel to maximize the performance and reduce overall job execution time with the use of parallelize component of Talend in TIS and using the Multithreaded Executions in TOS.
- Experience in reading and loading high-volume Type 2 dimensions by implementing SCD (Slowly Changing Dimensions).
- Extensively worked troubleshooting of Data Stage and Talend jobs, addressing production issues and fixing those on a timely manner. Performance tune & Enhance ETL jobs and SQL Queries as required.
- Hands on experience with common enterprise data quality subjects, such as data cleansing, de-duplication, data profiling.
- Good at writing UNIX shell scripts and commands.
- Extensive experience in Unit Testing, Integration Testing, System Testing, Regression Testing and User Acceptance Testing (UAT).
- Handled multiple projects by leading Offshore team. Effective in cross-functional and global environments managing multiple tasks and assignments concurrently with effective communication skills.
- Excellent team member with problem-solving and trouble-shooting capabilities, highly motivated, result oriented, ability to work cooperatively in a team and ability to grasp quickly.
TECHNICAL SKILLS
RDBMS: Oracle 11g,12c, MS SQL SERVER 2008, Netezza, Snowflake, Redshift
Languages/Scripting: C, C++, SQL, PL/SQL, HTML, CSS, JAVASCRIPT, UNIX Shell Scripting
Data Modeling: Star Schema & Snowflake Schema, Facts & Dimension tables, Erwin
Tools: ETL - IBM DataStage 8.7,9.1,11.5, Talend 7.0.1,7.3.1
BI - Tableau, QlikView 11.2 SR5: Scheduling - AutoSys, Control-M, TAC
Version Control - GitHub: Others - SQL Developer, Aginity, TOAD, JIRA, WinSCP, PuTTY
Operating System: Windows, Unix, Linux
Cloud Technologies: Trained AWS Solution Architect Associate.
PROFESSIONAL EXPERIENCE
Confidential, Northbrook, IL
Sr. ETL Developer
Environment: DataStage 9.1, 11.5, Talend 7.0.1, 7.3.1, QlikView, Netezza, Redshift, Snowflake, Control-M, TAC, Amazon Web Services (AWS), WinSCP, PuTTY, Windows 10, Unix
Responsibilities:
- Responsible for Requirements Gathering, Design and Development of ETL DataStage and Talend jobs.
- Extract, transform and load external source split-week data and Internal Source files to weekly, monthly and quarterly data marts to support Commercial Operations Landing Area (COLA) and Commercial Customer Master (CCM) Processes.
- Worked with Data Governance Team and external Vendors to understand the data provided by them and further design, Implement and support end to end application process.
- Developed complex ETL mappings for Staging, Dimensions, Facts and Data marts load. Identify and document data flow per dependencies after applying business rules.
- Created Talend Landing jobs to pull Data from Snowflake, AWS S3 Buckets, salesforce and load data into Redshift, Snowflake and Flat Files. Further Perform SDC Type-2 to maintain history in Transformation layer.
- Created Datastage Jobs and sequences to Pull Data from One application to other to process data through MDM HUB to create single record which will be generated as an Entity.
- Developed Datastage parallel and sequence jobs to Extract Transform and Load the data into Netezza database tables where Netezza database is logically split into DLZ (Landing), DTZ(Transformation) and PL (Presentation Layer) tables.
- Integration of Control M to schedule and monitor Daily/Weekly/Monthly/Quarterly jobs.
- Implemented Error Logging, Error Recovery, and Performance Enhancement’s & created Audit Process (generic) for various Application teams.
- Developed jobs to extract data from XML files and create Json and CSV files for creating S3 Data Lake.
- Developed generic/reusable jobs to bulk load data into Redshift Tables.
- Performed Validations and Quality Check's to make sure all Business rules are applied to the data.
- Performance tuning DataStage jobs and enhancing SQL code by analyzing the query plan.
- Configure central FTP repository to land internal and external files to be loaded into this data warehouse.
- Created generic Unix shell scripts which are used to call each datastage job and perform PRE and POST processing functionalities.
- Responsible to Create Design Specification document and Unit test Scripts to make sure ETL jobs and scripts are coded according to the business requirements.
- Production Support- Responsible for Fixing raised defects. Perform Unit and system integration testing.
- Use Repository Manager for Migration of Source code from Lower to higher environments.
- Schedule Talend Jobs in TAC through Job Conductor & Execution Plans in Dev, QA and Production environments to run jobs on daily, weekly, monthly and yearly basis (Cron Trigger).
- Troubleshoot issues with Nexus Artifact Repository to automate code deployment activities.
Confidential, Cleveland, OH
Sr ETL/BI Analyst
Environment: DataStage 11.5, Tableau, Oracle 12c, Unix, WinSCP, SQL Developer, Windows 10
Responsibilities:
- Worked with Business users to identify requirements and developing proof of concepts report.
- Handled two other Projects (Plant KPI, Canada Dealer Launch) by leading Offshore team.
- Coordinate with Business and Client to Resolve Critical Issues by making sure data and Reports are delivered on time.
- Coordinate with team members and administer all onsite and offshore work packages.
- Developed Parallel Jobs and Sequence jobs using various stages to load data into EDW Data mart for Supplier Spend Analytics project.
- Used various sources like Flat files, Excel Files, Kronos Files, XML, Salesforce, Data from Various Schemas to load data into Staging tables in Oracle Database.
- Worked on Automation of ETL Processes using DataStage Job sequencer. Create Sequences and Triggers to generate Surrogate key for the dimensions in the fact table for indexes and faster access of data.
- Analyzed performance and monitor work with capacity planning. Performance tuning of the jobs by interpreting performance statistics of the jobs developed.
- Created mapping documents for all the ETL Jobs (interfaces, Data Warehouse and Data Conversion activities).
- Developed Dashboard and prepare ad-Hoc Reporting. Utilized Tableau server to publish and share the reports with the business users.
- Documented ETL test plans, test cases, test scripts, and validations based on design specifications for unit testing, system testing, functional testing, prepared test data for testing, error handling and analysis.
Confidential, Charlotte, NC
ETL Datastage Developer
Environment: DataStage 9.1, MS SQL Server 2008, Oracle 11g, SQL*PLUS, GIRA, AutoSys, Windows, Unix
Responsibilities:
- Understand business processes and coordinate with business analysts to analyze user requirements.
- Created source to target mappings between sources to operational staging and targets.
- Involved in designing the procedures for getting the data from all systems to Operational Data Store.
- Design and develop sequence & parallel jobs to extract data from different sources which includes Oracle 11g, SQL Server, Flat files and apply business rules to transform the data, load them into files and RDBMS.
- Created parameter sets to group DataStage job parameters and store default values in files to make sequence jobs and shared containers faster and easier to build.
- Created Wait for File Activity jobs, Interface Notification and DataStage sequences to load the data into Target and send the success/failure reports to the end users via email.
- Implemented pipeline and partitioning parallelism techniques and ensured load balancing of data.
- Designed and developed backend PL/SQL packages in database layer, stored procedures, functions and triggers.
- Performed ETL Testing, debugging defects and monitored job runs using DataStage Director and AutoSys to ensure data loads are taking process from source to target.
- Performance tuning of DataStage jobs and the SQL used inside the job.
- Worked on Unix Shell Scripts to execute external programs and to retrieve internal ftp files.
- Used JIRA for Agile Development and to track defects.
- Responsible for Unit, System and Integration testing. Involved in UAT (User Acceptance Testing).
- Schedule and Monitor DataStage jobs using AutoSys on Daily/Weekly basis.
Confidential
Programmer Analyst
Environment: Oracle 11g-SQL, PL/SQL, DataStage 8.7, SQL Developer, TOAD, Windows, Linux.
Responsibilities:
- Generate DDL/DML scripts for creation of new database objects like tables, views, sequences, functions, synonyms, indexes.
- Involved in Creation and Maintenance of PL/SQL Packages, Functions, Stored Procedures, Shell Scripts for existing system as part of bug fixing and enhancements.
- Involved in debugging and Tuning the PL/SQL code, tuning queries, optimization for the Oracle database.
- Developed software test plans, test design specifications for various test scenarios.
- Understand existing business model and Translate Business Requirements into ETL jobs.
- Design and develop ETL jobs for extracting, transforming, integrating and loading data into data mart using DataStage Designer. Developed, executed, monitored and validated the ETL DataStage jobs in the DataStage Designer and Director Components.
- Use Designer to import Sources, design targets, create mapping and develop transformations.
- Worked in a team to construct data volumes mostly adhering to Star Schema using strong knowledge of Data Warehousing, Star Schema, Fact and Dimension Tables.
- Worked with various Partitioning techniques like Entire, Hash etc.
- Customized UNIX scripts as required for preprocessing steps and to validate input and output data elements, along with DataStage routines
- Worked with DataStage Director for setting up production job cycles for daily, weekly monthly loads with proper dependencies.