Sr. Big Data Quality Engineer Resume
Newark New, JerseY
SUMMARY:
- IT professional with approximately 8 years of experience spanning across multiple industries, namely retail, Banking and Finance Services and Insurance, with the substantial number of year being in Banking and Financial Services.
- Extensive experience in designing Data Quality Framework to perform data validation using Spark (pySpark), Spark SQL
- Expertise in writing python programs using Pandas/Numpy towards verification and validation of the raw data in HDFS environment
- Expertise in writing Hive Scripts and shell script
- Expertise at designing tables in Hive using SQOOP and processing data like importing and exporting of databases to the HDFS.
- Extensive experience in creating Tableau visualizations and dashboards for GAP analysis
- Expertise in analyzing data - validation of data quality, identify gaps in data with respect to requirements, prepare data sets towards verification and validation
- Worked actively across multiple databases towards writing complex SQL queries to extract data towards profiling leading to validation and verification
- Hands on experience with various ETL tools, such as IBM Datastage, Informatica
- Extensive Testing experience in Business Intelligence reporting with various BI tools, such as SAP Business Object, Tableau, SAS Reporting Tool, Oracle OBIEE Reporting Tool, Oracle Financial Services Analytical Applications (OFSAA)
- Extensive experience in authoring test strategy, test plan, test scenario and test cases and track the quality of the deliverable to the requirement
- Hands on experience in Agile - test driven development and feature driven development
- Go getter attitude with interest in techno-functional areas such as Anti Money Laundering, Fraud Detection and other Financial Crime related subjects.
- Strong analytical and problem solving skills, Good leadership and decision-making skills and Excellent inter personal skills.
TECHNICAL SKILLS:
Big Data Technologies: HDFS, Hive, HiveQL, Spark(pySpark), Python, Pandas, Numpy, Sqoop, Kafka, Hbase
Data Analysis: Teradata, Oracle, MS Access, MS SQL Server, DB2, Python
Defect Management: JIRA Agile, HP ALM, QC
Requirement Management: JIRA Agile, HP ALM
Business Intelligence: Business Object, Tableau, SAS Reporting Tool, Oracle OBIEE Reporting Tool, Oracle Financial Services Analytical Applications (OFSAA), Python Django
Modelling Technologies: Python, Pandas, Numpy
Programming/Scripting Language: SQL, PL/SQL, Unix Shell scripting, VB scripting, Python
PROFESSIONAL EXPERIENCE:
Sr. Big Data Quality Engineer
Confidential, Newark, New Jersey
Responsibilities:
- Subject matter expert for big data framework using Spark (pySpark), Sqoop, Hive etc
- Mentor and oversee implementation of various projects
- Architect Big data framework (BigRaft - Big Data relational automation framework for testing) using PySpark (Python and Spark) which helped in reducing 70% of execution time
- Participate in requirement gathering workshops to understand requirements towards strategizing the verification and validation process
- In depth data analysis - data extraction using complex SQL, establish data lineage and de-construct the transformation logic to understand the data flow vis-a-vis process
- Validate and verify user risk assessment predictive model to evaluate mortality risk life insurance underwriting process
Confidential, New York, New York
Responsibilities:
- Designed and prepared interactive and intuitive dashboards and reports using Tableau to conduct comparative analysis between current state and target state
- Extracted data towards reconciliation and created test data using complex Oracle SQL, PLSQL and Hive SQL
- Tested profiling of Customers and Accounts and computed their risk category based on various parameters
- Participated in strategy meeting towards best practice using Big data in alignment with Compliance and Anti-Financial Crime requirements
- Implemented seamless integration of JIRA and ALM using ORASI library
- Attended training workshops on Payment Systems and various methods to transfer funds
- Tested the logic to identify cross-border payments, country derivation logic, party identification logic
- Participated in daily stand-up meetings and scrum of scrum meetings to provide an update on what has been accomplished, what is planned for the current day and raise any impediments if any
- As a Team Lead, coordinated and managed the deliveries from offshore and provided regular updates to the client stakeholders and Confidential Delivery Manager
- Other key deliverables around Compliance Ml reporting, comprised of Confidential 239, Employee Breaches, etc.
Sr. Data Quality Engineer
Confidential, Phoenix, Arizona
Responsibilities:
- Validated data quality, data coverage and Identified gaps in data with respect to requirements
- Participated in requirement gathering workshops to understand requirements towards verification and validation strategy
- Participated in daily stand-up meetings and scrum of scrum meetings to provide an update on what has been accomplished, what is planned for the current day and raise any impediments if any
- As a value add, created utilities to alleviate manual overhead and promote cost effective measures in the field of both ETL and Bl reports being generated
- Good understanding of RWA calculation with experiences on exposures
- Validated equities exposures Cap Treatments, Risk Weight assignment/Non Significant treatments both for General Equities and Investment Funds Account
- Analyzed and burndown reporting of defect information to assist management with risk management, process improvement and project management
- Created reports across various management levels to help make the decisions and provide valuable suggestions in go/No Meetings
- Wrote complex queries using business Rules for Staging to Dmart to lmart
- Mentored new joiners in writing the complex SQL scripts
