Sr Data Engineer / Etl Developer - Big Data Apps Resume
Phoenix, AZ
SUMMARY:
- Experience in Project / Product development team in Agile - Scrum / Kanban Framework as Agile-Practioner/ Scrum Master in Large Scaled Scrum (LeSS) execution through Scrum ceremonies.
- Experience and strong knowledge in the areas of data Confidential, Modeling, Data integration (ETL), Data Quality, Data Governance and Data Security in EDW and Data Mart - Data Analytics & Reporting applications.
- Expertise in handling Business requirement gathering as BRD creation, Jira User stories, business use case / functional spec, JAD session with SME / cross functional application team and confluence document Confidential .
- Highly expertise in technical Data Analysis, Data Profiling, cleansing, data quality dimension rule check, data dictionary details / Metadata repository create and manage, business rules identification and implementation.
- Skilled in understanding and analyzing various data types like flat files, relational data, XML, Jason, image, pdf etc.
- Experience in using Informatica IDQ / Analyst tool for source data profiling, creating and applying rules, Process and Procedures in Define, Discovery, Profiling, Remediation, and Monitoring.
- Experience designing and executing Data Quality Audit/Assessment and data quality mappings that will cleanse to enrich data and standardize. Data quality scorecards to support data remediation process.
- Strong skill in writing complex SQL / PLSQL programs, Analytical functions, Joins, indexes, key constraints, batch process utilities, views, packages, stored procedures, triggers, functions and cursors etc.
- Expertise in OLTP and OLAP data modeling, SQL performance tuning techniques and normalization process.
- Design Semantic model structures, multi-layer data pipeline architecture in Big Data - Hadoop Data Lake process.
- Experience in Dimensional Model (Star / Snowflake Schema, OLAP Cubes) and Master / Reference model for MDM.
- Design and develop Logical and Physical data model, forward / Reverse engineering process in Erwin / ERstudio.
- Working with De-centralized / Data federation / Council in the Data Governance process and Audit support.
- Strong ETL experience in Informatica PC - Mapping Designer, Repository manager, Workflow Manager/Monitor.
- Well acquainted with Performance Tuning of sources, targets, mapping and sessions to overcome bottlenecks.
- Experience in resolving on-going maintenance issues and bug fixes; performance tuning of mappings and sessions.
- Design and development of ETL mapping in Ab Initio GDE to process data ingestion (HDFS)-Big Data Hadoop / Hive.
- Experience in UNIX shell scripting (file validations, file downloads, workflow executions).
- Experience in using Automation Scheduling tools - Control-M and Autosys.
- Experience in EBX 5 MDM tool in designing master / Reference data model, data integration process in cleansing, staging, merging. Creating workflow process, Rest API Services build for data InBound / Outbound with MDM.
- Expertise in building Executive Dashboard for KPI, Glossary, Digital Asset Confidential in EBX5 MDM.
- Created test strategy and test plans, test data for extraction and transformation processes and managed UAT.
- Experienced managing and driving all aspects of software development, resource Confidential and project Confidential . In depth expertise in project Confidential on large complex projects.
TECHNICAL SKILLS:
Databases: Oracle, Teradata, SQL Server, HIVE.
ETL /ELT: : Informatica PC / IDQ, Ab Initio, EBX MDM
Data Modeling: Erwin, ERStudio, Visio, MDM
Languages: SQL, PL/SQL, Java, XML, Shel, Python.
Big Data Frameworks: Hadoop HDFS, MapReduce, Hive, Pig, Sqoop, Flume, Ooziee. Kafka
Load/Unload Utilities: Teradata - BTEQ, Fast Load, MLoad, Tpump and Oracle SQL Loader
BI Reporting: Tableau, Trifacta, Cognos
Project Confidential: JIRA (Scrum / Kanban), Confluence, SharePoint
Scheduler: Control - M, Autosys
Data Collection: Rest API's, SQL
Others: SQL Developer, SQL Assistant, Toad Data Point / Apache, Squirrel SQL.
PROFESSIONAL EXPERIENCE:
Confidential, Phoenix, AZ
Sr Data Engineer / ETL Developer
Environment: Teradata, Hadoop - HIVE, Oracle, ETL - Informatica PC, AB Initio GDE / BRE, Unix, Control - m, JIRA-Scrum, Confluence, SharePoint, Bit bucketRoles and Responsibilities:
- Agile project Scrum team member as T-shaped skill of Technical Data Analyst / ETL developer in Big Data apps.
- Work with business stake holders in creating and grooming the product and sprint Backlog items in Jira.
- Creating Impact Matrix document for capturing the requirement details, creating the BRD / FSD documents, manage JAD session with SME’s, application team and business stake holders in business logic Sign-off.
- Work with product owners, business SME, application design architects on data ingestion and reporting architects to identify requirements and consolidate enterprise data model consistent with business processes
- Perform Data Analysis / Data mining from various source systems using SQL’s, capture metadata details, create data cleansing rules, Unwinding the logic from Stored procedures, ETL Mappings, SAS datasets etc.
- Manage JAD session with business and cross functional source application team in presenting and analyzing the data profile details with scorecard and capture data process business logic transformations.
- Create Source To Target data (DEM) mapping document with transformation logic for ETL build and data validation.
- Create Logical and Physical data model in multilayer process zone data architecture in STAR / SNOWFLAKE Dimensional modelling in Teradata database and Digital ECO system in HDP Hadoop HDFS - HIVE.
- Deep dive technical data analysis on the source systems to capture source data samplings and Metadata information on the sources as part of Data profiling, Data Cleansing and Data Quality process in Data Set.
- Analyze and reverse engineer the logic details of the DB Stored procedures, ETL mapping logic implemented in the source systems to capture the source objects for the application build.
- Manage performance tuning methodology in optimizing SQL’s, ETL mappings, HIVE- managed / ORC tables.
- Create sandbox, project folder structure and data directories for components in line with application architecture.
- Extracted the data from the flat files and other RDBMS databases into staging area and populated onto Data warehouse
- Design and development of ETL mappings in Informatica / Ab Initio GDE to process data ingestion in Hadoop HDFS / Hive and Teradata / Oracle database.
- Created sessions, configured workflows to extract data from various sources, transformed data, and loading to DW app
- Involved in Performance tuning at source, target, mappings, sessions, and system levels.
- Prepared migration document to move the mappings from development to testing and then to production repositories.
- Create, customize and fine tune load jobs in Control-M scheduler.
- Support for data governance program in Meta data analysis, business / data SME and MDR implementation.
- Manage the development team in technical solution build and support business in UAT and postproduction.
Confidential, Knoxville, TN
Lead Data Engineer / MDM Architect
Environment: Atlassian Jira - Scrum, EBX5 MDM Tool, Oracle 11g, SQL Server, Netezza, Confluence and SSIS ETL.
Responsibilities:
- Working as Architect / Scrum Master managing Agile - Scrum framework model project development.
- Work with business stake holders in creating and grooming the product and sprint Backlog items in Jira.
- Manage daily standup sprint meetings with Scrum team and work on resolving impediments causing roadblocks for team members in delivery. Conduct Sprint planning and sprint retrospective meeting for release efficiency.
- Interaction with Business Users / Stakeholders in understanding the Business Requirement, Creating the detail BRD design document, handle meeting for the presentation and Sign-off with BRD.
- Architect the application and data structure design and process flow rules with business entities.
- Initiating and Managing JAD sessions with Business Analyst / SME’s / Application team on business rules functionality logic documentation and create FSD with details on data build and Data Governance.
- Create HLD and LLD design documents to capture the Business and Functional Use Cases, Application design methodology and Data Flow process diagrams details.
- Design Logical Data model structure by identifying Entities, Attributes and their cardinalities entities.
- Create detail Physical Data Model design document and discuss the model details with Data Architect team on the implementation plan and maintain the documentation library.
- Deep dive technical data analysis on the source systems to capture source data samplings and Metadata information on the sources as part of Data profiling, Data Cleansing and Data Quality process in Data Set.
- Analyze and reverse engineer the logic details of the DB Stored procedures, ETL mapping logic implemented in the source systems to capture the source objects for the application build.
- Generate the Physical Data model in the EBX5 MDM in align with associated Domain and Dataspace.
- Create the Source-To-Target technical mapping specification document with SQL’s logic for the ETL / Custom programs to implement the business validation and the data load process.
- Developed the Digital Asset Confidential to capture unstructured data like .jpg, pdf, excel etc., in EBX5 MDM.
- Created the Dashboard for KPI’s for the executive Confidential and business users on the application, and also implemented the Business Glossary and metadata details in EBX 5 for Users.
Confidential, Phoenix, AZ
Lead Data Analyst / ETL Lead Developer
Environment: Exigen Insurance Product, Oracle 11g, SQL, PL/SQL, Informatica PowerCenter, IDQ, PL/SQL Developer, Agile - JIRA, HP QC, ALM and MS SharePoint.
Responsibilities:
- Representing as Lead Data Analyst / Leading the Analyst team in the Business Requirement Analysis, Data model design, Technical Data Analysis, Data mapping documentation, ETL build and validation.
- Interaction with the Business Users / SME / Applications Developers in analyzing the Requirements and Creatingthe LLD and HLD documents capturing Business / Functional Use Case, Data Flow Diagram and process flow details.
- Initiating and Managing the JAD sessions with Business Analyst / SME / IT team on the business rules signoff.
- Detailed analysis of Source Data models & Data Structures of both Source (As-Is) and Target architecture (To-Be).
- Handle the Technical discussion phase discussion with Data Architect team on Data model design approach.
- Perform Deep Dive Data Analysis / Profiling, Data Quality process in identifying Source systems and capture the Meta data information, data structure / consistency, gap analysis details.
- Initiate and perform Data cleansing process on the source data to build the Target system based on requirements.
- Design and Create the Logical Data Model to capture the business entities, attributes and their cardinalities.
- Transform the LDM to Physical data model design in the ERWin data model tool and generate the DB Schema.
- Create the Technical Specification STT Data Mapping document with the data transformation logic details with Business rules for ETL Mapping development and data validation purpose.
- Analyzing and Developing Complex SQL queries, Stored Procedures, ETL Mapping for application development.
- Production support on the Adhoc reports for the Business users thru SQL logic with SLA process.
- Implementing the tuning process for the long running queries on the canned reports thru SQL optimization.
- Data migration using Bulk data processing technique, SQL Loader utilities and External table features.
- Developed and optimized complex queries for data retrieval and query tuning using Explain plan, SQL Trace and TKPROF Utilities for SQL optimization techniques to maximize performance.
- Handle data Partitioning design methods and Indexing techniques on the application.
- Constantly interact with DMC (Data Confidential Council) & Collaborate with application SME’s to identify and thoroughly validate the flow of data elements, its integrity across the LOBs.
- Handle meetings to review with SME and Business leads and bring to an agreement/sign-off on mapping and requirements within a defined timeframe.
- Handle team in a multicultural/virtual team environment and Onshore / Offshore delivery model.
Confidential, Dallas, TX
Lead Data Analyst / ETL Developer
Responsibilities:
- Represent in the team as Data Analyst / Leading the Analyst team in Business Requirement Analysis, Design document, ETL and BI Report Development, Testing, Implementation and support phase of application.
- Interaction with the Business Users / SME / Applications Developers in analyzing the Requirements and Creating Business Requirement (BRD) and Functional Specification document (FSD).
- Initiating and Managing the JAD sessions with Business Analyst / SME / IT team on the business rules and logic.
- Analyzing the detailed Meta Data information on the Source Systems and create detail analysis document.
- Detailed analysis of Source Data models & Data Structures of both Source (As-Is) and Target architecture (To-Be).
- Deep Dive Data Analysis / Profiling, Data Cleansing, Data Quality analysis process to build Technical Transformation details specification for Application Development.
- Creating HLD and LLD Specification document for the design preparation for ETL and Data Architect team.
- Build Data element mapping logic for Source to Target data load, Create mapping specification document.
- Involved in the Architecture design technique, Create Logical and Physical Data Modeling using ERWin tool.
- Constantly interact with DMC (Data Confidential Council) & Collaborate with application SME’s to identify and validate the flow of data elements, its integrity across the LOBs.
- Analyzing and Developing Stored Procedures, SQL Scripts, ETL Mapping for application development.
- Data migration using Batch processing, SQL Loader and External table and Teradata utilities FastLoad and MultiLoad
- Developed and optimized complex queries for data retrieval and query tuning using Explain plan, SQL Trace and TKPROF Utilities for SQL optimization techniques to maximize performance.
- Handle data Partitioning design methods and Indexing techniques on the application.
- Creating ETL mappings and enhancing existing mappings to facilitate the data load in system.
- Review with SME and Business leads and bring to an sign-off on mapping and requirements within defined timeframe.
- Managing the team in development phase, data validation, code review and defect analysis to fix issues.
- Developing jobs scheduler and Shell Scripts for the ETL job automation in UNIX environment.
- Handle team in a multicultural/virtual team environment and Onshore / Offshore delivery model.
Environment: Oracle 11g, Teradata 14.x, SQL, PL/SQL, Informatica PowerCenter, IDQ, Toad, PL/SQL Developer, Teradata SQL. Assistant, SSH Client, UNIX Scripting, Control - M, HP Quality Center, PVCS and MS SharePoint.
Confidential, Bristol, CT
Sr. Business Data Analyst
Responsibilities:
- Interacting with users to capture, analyze business needs and user’s strategic information requirements.
- Creating high level and low level functional specification and Technical specification design document.
- Analyzing and creating the Logical DM model and Physical DM with modeling team in the application development.
- Develop and Enhancement of DB objects like Packages, Procedures, Functions, triggers, ref-cursors, views, Collections etc. in the application development.
- Developing and optimizing existing SQL queries in the application using Explain plan, SQL Trace and TKPROF Utilities to improve efficiency.
- Data migration using Oracle Batch processing, SQL Loader and External table techniques.
- Handling and Managing data partitioning design methods and indexing techniques in the application.
- Develop UI using Oracle Forms 10g/6i, Oracle Reports 10g/6i and SQR Reports for the web enabled application.
- Creating functional and technical ETL mapping specification document for data mappings.
- Develop/support ETL transformation mapping and enhancing existing mappings to facilitate data load into DWH.
- Handled Slowly Changing Dimensions techniques in the ETL developments (SCDs, Type I, II& III).
- Implemented performance tuning logic on Targets, Sources, mappings and sessions to provide maximum efficiency.
- Creating BI reports using Cognos ReportNet using Query Studio and Report Studio.
- Handling team on the development phase, code review and analyzing programs to identify and fix issues.
- Developing Shell Script programs for the job automation in UNIX environment.
- Handling application release activities for UAT phase and production release for the application users.
- Providing production support as needed offering timely, instantly sound problem resolution.
- Create, maintain and tracking of end users business issues in application.
Environment: Oracle 10g, SQL, PL/SQL, Informatica PowerCenter 8.x, Oracle Forms 10g/6i, Reports 10g/6i, SQR Reports, Cognos Report Net 8.x, UNIX Scripting, Mercury Test Director, SSH Client, Citrix, PVCS Config, MS SharePoint portal.
Confidential, Boston, MA
DW Lead Developer
Responsibilities:
- Involved in Business requirement analysis and creating functional specification design document.
- Data Model design for OLTP app. and Dimensional model design for the Data Warehouse application development.
- Development and enhancement of DB objects like Stored Procedures, Functions, Triggers and advance features.
- SQL Performance tuning to increase operational efficiency using Explain Plan and TKPROF Utilities.
- Handled data migration using SQL*Loader and External Tables concepts to load data from external system and developed PL/SQL programs to dump the data form staging tables into base tables.
- Handled Oracle batch processing using Bulk techniques for performance efficiency.
- Implementing the advance features of oracle like Oracle Streams and Oracle AQ methods in the development.
- Handling data partitioning and indexing, Keys, constraints and data model design techniques with DM team.
- Developed ETL Specifications and Mappings using Informatica PowerCenter tool for Data loading.
- Experience in code review and analyzed the PL/SQL programs to identified bugs and bottlenecks.
- Created detailed project outlines, application design specifications, and test plans.
- Designed and developed Oracle Forms & Reports applications using Forms 9i/6i and Reports 9i/6i.
- Developing and maintain BI canned and Adhoc reports for business in Cognos.
- Provided on-call production support for user Applications, Involved in system unit testing and Integration testing.
Environment: Oracle 9i/10g, PL/SQL, ETL Informatica PowerCenter 8.x, Oracle Forms 10g/Reports 10g, Oracle 9i AS, Cognos ReportNet, UNIX Solaris, Mercury TestDirector, MS Sharepoint portal, Accurev Configuration Confidential .
Confidential
Senior Systems Analyst
Responsibilities:
- Understanding business requirement needs and creating application design.
- Defined and created entities, functions and flows between entities.
- Created logical and physical database modeling design and database scripts.
- Developing PL/SQL programs to implement business logic using Triggers, Stored Procedures and SQL Scripts.
- Performance tuning to increase efficiency of the system using SQL optimization techniques.
- Data migration activities using SQL*Loader, External tables to load data from external system.
- Designed and developing User interface using Oracle Forms & Reports.
- Extensive use of oracle forms and reports features like object groups, libraries, LOV and different canvases.
Environment: Oracle 9i, PL/SQL Developer, TOAD, Erwin 3.5, SQL*Loader, ETL Informatica PowerCenter, Cognos Report Net 8.x, Oracle Forms 9i/10g, Oracle Reports 9i/10g, Mercury Test Director.