Sr. Etl Developer/data Engineer Resume
Monterey Park, CA
SUMMARY:
- Seasoned Sr. ETL Developer/Data Engineer with 8.3 years of Software Development (ETL/Business Intelligence) experience in focusing on Data warehousing, Data modelling (3NF, Dimension, ER), Data integration, Data Virtualization, Data Migration, ETL process and Business Intelligence in Banking domain on various commercial and retail Lending Products .
- Strong experience in Architect, Designing, Building and Implementing Data Warehousing/Data Virtualization/Business Intelligence Products/Prototypes/POC using Informatica Power Center 8.x/9.x/Power Exchange with Mainframe/SQL Server, Informatica Data Services and Business Objects.
- Highly skilled on using effective SDLC methodologies like Waterfall & Agile/Scrum based upon Business Requirements, Implementation Schedule and Resource availability.
- Drawing on Experience in all aspects of Software Development (Data Warehouse/Business Intelligence) solutions (Database issues, Data modeling, Data mapping, ETL Development, metadata management, data migration and reporting solutions). I have been a key resource in delivering innovative DW - BI solutions to the Confidential for Credit Risk Reporting.
- Knowledge of Key Credit Risk Reports/Calculations like Allowance for Credit Losses (ACL), Loss Factors, Balance Control Summary, Delinquencies, Charge-off, Credit Policy Exception and various other Loan Level Reports.
- Expertise in large scale data integration as part of Confidential ’s multiple Acquisitions .
- Strong Experience on working with Business/Data Analysts and other Technology Partners to identify and understand Reporting/Data requirements and translated them into High Level and Detailed (Source to Target) Design documents.
- Elicitated requirements from End Users/Stake Holders and translated them into High/Low Level Design Documents & ETL Programs to deliver the same.
- Possess strong and trust worthy relationship with Business and Technology Partners by working hand in hand and delivering quality solutions .
- Proficient in interaction with the business users by conducting meetings throughout the entire SDLC process to ensure smooth delivery of solutions.
- Well versed in developing and understanding the complex SQL queries, unions and multiple table joins and experience with Normal/Materialized Views.
- Strong knowledge of Data Architecture using Entity-Relationship modelling, Facts and dimensions tables, slowly changing dimensions and Dimensional Modeling (Star Schema and Snow Flake Schema).
- Highly Skilled on writing SQL queries to validate data, Fixes, Data Quality Issues etc.
- Quality coding skills that promotes reusability, maintainability and good, scalable performance.
- Extensively worked on Data extraction, Transformation and Integration from various sources like Oracle, SQL Server, DB2 and Flat files.
- Extensively involved in Optimization and Tuning of mappings and sessions in Informatica by identifying and eliminating bottlenecks, memory management and parallel threading.
- Expertise in Business Model development with Dimensions, Hierarchies, Measures, Partitioning, Aggregation Rules, Time Series, Cache Management.
- Extensively worked on data extraction, Transformation and loading data from various sources like Oracle, SQL Server and Flat files.
- Experience in creating Reusable/Non Re-usable Transformations (Joiner, Sorter, Aggregator, Expression, Lookup, Router, Filter, Update Strategy, Sequence Generator, Normalizer and Rank), Mapplets and Mappings using Informatica Designer and processing tasks/Sessions/Worklets/Workflows using Workflow Manager to move data from multiple sources into targets.
- Strong Experience in evaluating/POC MPP platforms like Exadata, GreenPlum and Data Virtualization Engines like Informatica Data Services.
- Highly skilled in writing and executing Efficient and effective Test Strategy, Test Cases and Test Scripts to ensure quality data.
- Experienced in writing shell/Unix scripting on Unix/Linux platforms to call ETL jobs, to perform file transfers using FTP & SFTP .
- Highly skilled in scheduling ETL jobs using Shell Scripts on Industry standard Job Scheduling Tools like Autosys and Tivoli.
- Strong Experience in Defect Management/Application Life Management using HP Quality Center/ALM.
- Hands on Experience in Informatica Administration involved in Admin Console, Repository Manager, User & Group Creation, Identify Session Locks and Killing Long Running Session and performing Upgrades when moving from current version to latest version.
- One of key expert of the existing Confidential Data warehouse having approximate size of 20 TB data and Credit Risk data marts of around 3-5 TB. Data warehouse hold more than 700 tables with combination of CDC of 1 and 2 and Data Mart has around 300 tables.
- Passionate about leaning and implementing new Software Development technologies/products like Big Data, Hive, Python, Hadoop, Composite, MDM etc.
- Via Repository Manager done Export, Import, Copy Metadata Objects and Reports through Metadata Repository Database.
- Experience with coordinating and leading onsite-offshore development .
- Excellent team player with very good communication, Inter Personal/People and leadership skills/qualities.
- Experience in working with cross-functional teams .
TECHNICAL SKILLS:
ETL Tools: Informatica Data Services 9.x (Data Virtualization), Informatica Power Center 7.x/8.x/9.x, Informatica Power Exchange 8.x
Reporting Tools: Learner on Business Objects XI 3.1
Programming Language: SQL, PL/SQL, Shell/Unix Scripting, C++, ANSI SQL
Job Scheduling Tools: Autosys, Tivoli Workflow Scheduler
RDBMS: Oracle 9i/10g/11g, SQL Server 2008 R2/ 2008/2005, 7.0, MS-Access, DB2, Postgres, DB2
Meta Data Management: Meta Center
Data Modeling: Logical Modeling, Physical Modeling, Relational Modeling, ER Diagrams, Dimensional Data Modeling (Star Schema Modeling, Snowflake Schema Modeling, FACT and Dimensions Tables), Entities, Attributes, Cardinality, MS Visio
MPP Platform: GreenPlum 1.16.1
Version Management: Microsoft Integrity Client, Team Foundation Server
Operating System: Unix/Linux, Windows
Data Quality: HP Quality Center
Incident Management: HP Service Manager (Peregrine)
Project Management: Microsoft Project Planner
Other Tools: TOAD 9/10.0, pgAdmin 1.16.1, SSH Client, SQL workbench, Visio, Microsoft Office Suite, WinScp
PROFESSIONAL EXPERIENCE:
Confidential, Monterey Park, CA
Sr. ETL Developer/Data Engineer
Responsibilities:
- Interact with the Business users to identify and document Business and Data requirements for software development.
- Develop FRD (Functional requirement/Design Document) and data architecture document by working closely with Business Community and other technology partners using Agile and Waterfall SDLC methodologies.
- Responsible to conduct Impact and feasibility analysis on current and future pipeline projects on ICRS data mart.
- Scrutinize functional requirements, specifications and source to target mapping documents, identifying underspecified requirements and other design challenges in order to ensure smooth SDLC processing to support business strategy and deliver quality business reporting.
- Develops detailed analysis, design (Source to Target Mapping Specifications), construction and testing specifications ensuring technical compatibility and integration.
- Create data flow diagrams, System Architecture Diagrams, data mapping documents from Source to stage and Stage to Target mapping documents indicating the source tables, columns, data types, transformations required and business rules to be applied.
- Responsible for all driving SDLC activities related to architecture, design, development, implementation, administration and support of ETL processes for large scale data warehouses using Informatica Power Center 8.x/9.x and Oracle by taking other team members together.
- To Perform Source system analysis (SSA) to identify the source data that needs to be moved into the target tables.
- To Ensure Test Driven Software Development by Creating Test Plans, Test Cases, and Test Scripts for all testing events such as Unit Testing (DUT), System Integration Testing (SIT), and User Acceptance Testing (UAT).
- Ensure Quality Software Development using standard and mappings and with various transformations like Expression, Aggregator, Joiner, Router, Sorter, SQL Transformation, Normalizer, Lookup (Connected and Unconnected) and Filter using Informatica Power Center 8.x/9.x.
- Create Reusable and Common Transformations/Mapplets (Joiner, Sorter, Aggregator, Expression, Lookup, Router, Filter, Update Strategy, Sequence Generator, Normalizer and Rank) and Mappings using Informatica Designer and processing tasks/sessions/worklet using Workflow Manager to move data from multiple sources into targets using Informatica Power Center 8.x/9.x.
- Responsible for creating data base objects like Tables, Synonyms, Indexes, Views, Triggers, Stored Procedure, Functions, Grants using SQL using Toad.
- Validate and reconcile GL and LOAN data on daily basis using complex SQL and automated application (BRC) on daily basis.
- Create and Implement Confirmed Dimension, FACTS, Aggregate, Rules and Static Tables as part of Dimension Modelling using Star Schema.
- Assist BI Team in analyzing and resolving reporting data issues, hand=on experience in developing adhoc repots using Business Objects.
- Implement Change Data Capture Logic (CDC-1/2/3/6) by tracking the changes in critical fields required by the user and demanded by data model using Informatica Power Center 8.x/9.x.
- Perform Unit and Peer Code reviews for offshore team members to ensure accuracy.
- Extensive use of Persistent cache to reduce session processing time.
- Responsible for defining and Maintaining best practices, Coding standards and review checklist for Informatica Power Center 8.x/9.x and Oracle Development for onshore and offshore team members.
- Create Workflows/Worklets using various tasks like Email, Scheduler, Control, Decision, and Session in the workflow manager.
- Made use of Post-Session success and Post-Session failure commands in the Session task to execute scripts needed for cleanup and update purposes.
- Responsible for maintaining data lineage for all KDE (Key Data Elements) in meta center
- Perform root cause analysis for production issues and identify the short term and long term fix (SQL or ETL) in limited amount of time using Toad and Reporting Tool (Business Objects).
- Support Production Environment on 24/7 basis.
- Resolving issues related to Enterprise data warehouse (EDW), stored procedures in OLTP system and analyzed, design and develop ETL strategies.
- Implement automated file transfer process using Secure and Non Secure file transmission methods like SFTP and FTP.
- Creating Validation SQL for Data comparison and equivalence as per client Requirement using Toad.
- Using UNIX shell script to send mail if there is any failure without any Manual intervention.
- Troubleshooting of long running sessions and fixing the issues related to it.
- Creating Deployment group whenever we have project release. Used to move the code from one environment to another environment, cleansing the unwanted files as part of Informatica Administration.
- Support application development through system testing and user acceptance testing, bridging gap between business rules and technical implementation when necessary
- Defect management using HP Quality Center/ALM.
- Development of Shell Scripts to evoke the ETL jobs via Autosys Scheduler.
- Job scheduling using TWS and Autosys schedulers using shell scripts.
- Involved in Performance/ Query tuning. Generation /interpretation of explain plans and tuning SQL to improve performance using session partitioning, SQL override tuning, persistent caching, session re-arrangement.
- Involved in admin tasks like analyzing Table space requirement, load balancing and performance.
- Mentoring and assisting new joiners and existing team members by preparing Knowledge Transfer documents and Group meetings.
Confidential
Data Virtualization Architect
Responsibilities:
- Working with business users and business analyst to elicitate Business and Data requirements and perform requirement analysis.
- Translate business requited to high level and low level design documents and prepare S2T documents for team to write the data extraction programs using Informatica Data Services (Data Virtualization) using Agile/Scrum SDLC Method.
- Developed Logical and Physical data models that capture current state/future state data elements and data flows.
- Identify Source, Stage, Cleansing, Transformation and Presentation layer as part of Data Virtualization Framework.
- Use Informatica Data Services (Data Virtualization) to create the physical data objects, Logic data objects, logical data models, Mappings, workflows, reusable transformations, SQL data Services and applications.
- Responsible for maintaining Data Integration services and Caching for IDS objects.
- Responsible for maintaining the repository and integration services.
- Setup data services and ODBC connections for users to use the SQL data services for data analytics in Microsoft Excel, Business Objects, Toad Data Point as part of Administration activities.
- Creating Database (Oracle, SQL Server, and ODBC) connectors on DIS or Informatica data services.
- Develop optimized IDS programs to achieve push down optimization for better performance.
- Maintaining Logical Data Object Caches on Oracle Database, Setting Up Caching Interval etc.
- Ensure Load Balancing on DIS by monitoring long running and unwanted processes from multiple users as part of Administration activities.
- Write Shell/Unix Scripts to call the DIS jobs on Autosys to perform automation for data extraction.
- Apply and refine established ETL development standards and coordinating teams of ETL developers (typically 2 to 5) across multiple projects.
- Mentoring and assisting new joiners and existing team members by preparing Knowledge Transfer documents and Group meetings.
- Perform code reviews to ensure quality coding and data.
- Develop and implement processes to improve and increase project efficiency.
- Validates and Review the Test Plans and Test Scenarios, Test Cases; verify Test Results to ensure test driven software development.
- Performance tuning of IDS objects in case of any bottlenecks.
- Defect management using HP Quality Center.
Confidential, CA
ETL Developer
Responsibilities:
- Used Informatica Power Center for (ETL) extraction, transformation and loading data from heterogeneous source systems into target database.
- Created mappings using Designer and extracted data from various sources, transformed data according to the requirement.
- Involved in extracting the data from the Flat Files and Relational databases into staging area.
- Mappings, Sessions, Workflows from Development to Test and then to UAT environment.
- Developed Informatica Mappings and Reusable Transformations to facilitate timely Loading of Data of a star schema.
- Developed the Informatica Mappings by usage of Aggregator, SQL overrides usage in Lookups, source filter usage in Source qualifiers, and data flow management into multiple targets using Router.
- Created Sessions and extracted data from various sources, transformed data according to the requirement and loading into data warehouse.
- Developed several reusable transformations and mapplets that were used in other mappings.
- Involved in Unit Testing and Resolution of various Bottlenecks came across.
- Implemented various Performance Tuning techniques.
- Created PL/SQL Stored procedures and implemented them through the Stored Procedure transformation.
- Used debugger to debug mappings to gain troubleshooting information about data and error conditions.
- Implemented the Incremental loading of Dimension and Fact tables.
- Develop, test and implement break/fix change tickets for maintenance.
- Created sequential batches and concurrent batches for sessions.
- Developed Interfaces using UNIX Shell Scripts to automate the bulk load & Update Processes using scheduling tools like Tivoli and Autosys.
- Execute Test Cases and Capture Test results.
- Migrated mappings/sessions/worklets/workflows from Development to Testing and from Testing to Production.
- Developed local and global parameter files for ETL programs to support ETL batches for dynamic execution.
- Created various Documents such as Source-to-Target Data mapping Document, and Unit Test Cases Document.