We provide IT Staff Augmentation Services!

Big Data Engineer Resume

SUMMARY

  • Over all 13 years of experience in Data Warehouse applications, BI solutions, ETL migration projects, Cloud projects (using Azure and Hadoop Big Data Technologies), and GUI applications
  • Strong SQL programming experience in Netezza (PDA), SQL Server, Teradata, Oracle
  • Developed efficient enterprise data integrations between heterogeneous sources and targets using SQLs/ ETL tools/ Unix Scripts/ Python and automated the application workflows
  • Proficient in analyzing requirements and designing/building Common ETL Frameworks for Extracting / Loading Tables
  • Strong domain experience Health Care Provider, Insurance/Payer and Banking Regulatory Capital Calculations and Compliance Applications
  • Experience in building healthcare solution and Datalake project in Microsoft Azure services - ADLA (U-SQL), Azure Data Lake Store (ADLS), ADF, U-SQL,SQL SERVER, Python, Power BI
  • Experience in building healthcare Big Data project using Hortonworks (Hadoop), HDFS, Spark, Python, HBASE, Phoenix, Sqoop technologies
  • Experience in developing and validation of ETL data transformation using Star and Snow Flake Schema, history and incremental data load (SCD logic), BI Reports / Visualizations
  • Healthcare domain experience - Provider Related Opportunity Analysis(Cost, LOS, Care Monitoring), Appointment/Slot Utilization, Ambulatory and Hospital Extract Process, Professional/Hospital Billing, Medicare Pharmacy Claims, PBM, Medical Claims, Disclosures, Medical Management, Lab Results, Disease Management
  • Banking domain experience in Economic & Regulatory Capital calculations which include BASEL I, II, III (Credit & Market Risk), RWA, EC calculations and FFIEC 101 reporting, Risk Management, Consumer Credit Card (EU and USA)
  • Strong experience in developing software Test Plans and Strategies and Defect Management Process
  • Experience in Automating Validation Framework using QTP, SQL, U-SQL, Python Scripting
  • Exposure to AWS S3, Athena, RedShift services

TECHNICAL SKILLS

  • SQL (SQL Server, Netezza(PDA), Teradata, Oracle), U-SQL, Python
  • DataStage, Alteryx
  • Microsoft Azure- ADLA, ADLS, U-SQL, ADF, SQL SERVER, Powershell
  • Hortonworks - Hadoop, HDFS, PySpark, Sqoop, HBase, Phoenix, Linux
  • Power BI, QlikSense, Tableau
  • JIRA, VersionOne, Confluence, VSTS, HP ALM

PROFESSIONAL EXPERIENCE

Confidential

BIG Data Engineer

Responsibilities:

  • Analyzing vender extract layout and requirement document, and working closely with Clarity SME/ Business owner to identify the column mapping.
  • Developing loading scripts and configuration to copy data from Clarity (EPIC - EMR system) to Stage database (Netezza/PDA)
  • Developed complex SQL scripts to extract data from Staging layer tables to Extract tables
  • Built ETL/SQL scripts to meet the requirement for Health Care Ambulatory (Out Patient, Emergency) and Hospital (In-Patient Visit, Surgical) layouts
  • Following are the few layouts built through ETL process - Encounters, Biometrics, Medications, Procedure, Lab, Diagnosis, Active Problems, Past Surgical and Medication, Provider, Patient, Discharge and Admission Encounters Details
  • Created jobs using existing DataStage frame work, updated configuration to read the SQL scripts and generated extract table and file
  • Worked in multiple vender project and developed ETL scripts based on specific business use cases
  • Coordinated with Business Owner and Vender to get their feedback and make changes based on feedback
  • Validating Extract File output using Data Profiling Techniques, ensuring data quality and data consistency

Confidential, Radnor, PA

Big Data Engineer

Responsibilities:

  • Worked in Healthcare Provider Solution, developed in Azure Cloud Environment, using Azure Data Lake Analytics (U-SQL), Azure Data Lake Store, Azure Data Factory (Pipeline), SQL Server, Python, Power BI, Power Shell Scripts
  • This solution delivers Healthcare Provider Related Opportunity Analysis(Cost, LOS, Care Monitoring, Resource Utilization) with Benchmark data
  • Participate in Story Grooming, Effort Estimation, Risk Assessment
  • Developed U-SQL to transform the data from staging layer to MART layer following the data model and S2T mapping document
  • Automated the workflow of different U-SQL using Azure Data Factory
  • Built Datalake (ADLS) structure to upload data from different venders for different subject areas - Encounters, Charges, Provider, Pharmacy, CDM and then automated data copy using ADF
  • All configurations are done through JSON files and called in through Python scripts.
  • Reconciling data between new Azure solution and legacy applications (SQL server based), and perform root cause analysis on differences
  • Automating the regression test suite using U-SQL, ADF, Python scripts.
  • Automated Data Profiling, Data Reconciliation and Key Business Rule validation through Alteryx and SQL Queries for another solution developed in SQL server
  • Exposure to AWS S3, Athena, RedShift services. Supported team in identifying AWS vs Azure cloud services

Confidential

ETL / DW, BI Test Lead

Responsibilities:

  • Objective of this project to decommission existing mainframe application and process data through new EDW system
  • Requirement analysis and understanding of DW/BI requirement, mapping documents and data model
  • Effort estimation, resource planning and coordination with Project Manager
  • Develop test strategies, test plan/test approach and testing artifacts for DW/ETL and BI system
  • Testing ETL applications which includes data from Medicare Pharmacy Claims, PBM, Medical Claims, Disclosures, Medical Management, Disease Management
  • Create test scenarios and conducts reviews with development and BSA team
  • Analyze and understand the ETL work flows developed and perform End to End data validation using MLOAD, FLOAD, UNIX, SQL Scripts and Data Stage Jobs
  • Writing validation scripts to ensure incremental (SCD logic), FULL, History load is developed correctly
  • Write SQL queries (Teradata, DB2) to validate complex business transformation and lookup rules
  • Analyze BI business requirement and perform data and format validation for reports developed in SSRS
  • Prioritize and Execute System (SIT), Integration, Regression, E2E testing
  • Co-ordinate with offshore/onshore testing team (4 members) to perform all phases of SIT testing
  • Perform data analytical, reconciliation testing and support BSA for UAT testing

Confidential

ETL and BI Onshore Test Lead

Responsibilities:

  • Worked in application that calculates Risk Weighted Assets, Economic Capital numbers based on FINRA regulations (BASEL II & III Rules). This application gets data from multiple upstream applications which source enterprise data such as Wholesale (TCP, CRM), Retail (Mortgage, QRE), Traded Products (Repo, Derivatives), Securitization and Standard Charges at exposure/transaction level from different legal entities.
  • Worked in application Regulatory Capital reports (such as FFIEC 101) and internal data control reports are being developed and generated on monthly basis and validated for data accuracy and format standards
  • Partnering with BAs, application teams, project managers, and others to sprint planning, grooming, test planning, defect management activities.
  • Leading 11 member testing team (2 onshore and 9 offshore resources) on different applications which include ETL logic validation and Business Intelligence (BI) Reports created for regulators
  • Participate in sprint priority meeting and estimate the story for each sprint/iteration
  • Participate in grooming, daily standup, break-out meeting and interact with Developer, BA, PO to ensure story objective is meet and defects are communicated and fixed (Agile development process)
  • Building complex SQL statements involving all join variants to enable them to make robust comparisons of source versus target data loaded by the ETL
  • Validating 30 newly developed regulatory reports created in Microstrategy, Tableau,Cognos, Actuate against business requirement
  • Reconciling data between old and new system and provide explains for differences
  • Creating automated test scripts for data validation using SQL and stored procedures
  • Prepare requirement for test data and condition test data required for slowly changing dimension (SCD) testing.
  • Defect tracking, defect reporting using Quality Centre, VersionOne(Agile Tool)
  • Creating test plan for multiple application and coordinating with project manager for approval
  • Identify testing risks and issues and escalate appropriately
  • Analyzing business and functional requirement to design System Integration and User Acceptance Test Cases
  • Defining scope for regression testing and updating regression suite based on latest enhancement every sprint
  • Identifying automation scope for regression testing and coordinating with centralized automation team
  • Validation of BI reports generated through ACTUATE tools

Confidential

Test Lead

Responsibilities:

  • Worked in following Enterprise complaints applications: MDM, ISSUE TRACKER, EXAM TRACKER, LINSS, MIS, ERIM, MDM BOV, GBAM
  • Worked in 7 consumer credit card applications (RMS, CTA, LPW, LPFA, CORE, OMNI & FDMI and LVIS) which deals with Fraud Detection, Fraud Analysis, Delinquent Customer Accounts, Recovery Management system, Collection & Tracking, Payment Settlements, and Reage Process
  • Worked in both European and USA CC applications, and end to end Delinquency Process
  • Coordination with SME and client to gather legacy application requirement and knowledge transfer to team member
  • Leading 8 member offshore team, and coordinating testing activities such as test plan, test scripts, execution plan, regression scripts, and release metrics
  • Weekly and Daily test execution status email to both onshore and offshore manager
  • Defect Management through Quality Center and coordination with developers to defect closure through defect triage meeting
  • Meeting testing SLA provided by client for all testing projects
  • Generating Quality, Resource Utilization, Process improvement related metrics
  • Automation of regression scripts through QTP 9.5
  • Environment: Mainframe - QWS3270 Emulator, Web Technology, Quality Center 9.2

Hire Now