Big Data/hadoop Tester Resume
Chicago, IL
SUMMARY
- Over 6 years of professional experience in working with projects driven on Microsoft SQL, Data Warehousing and ETL processes.
- Experience with Software Development Life Cycle (SDLC), Agile, Scrum and Project Management Methodologies.
- Exclusive experience in all aspects of Software Test Life Cycle including System Analysis, Design, Development, Execution, Reporting and Closure Documentation.
- In depth knowledge of Testing methodologies, concepts, phases, and types of testing, developing Test Plans, Test Scenarios, Test Cases, Test Procedures, Test Reports and documenting test results accordingly after analyzing Business Requirements Documents (BRD), Functional Requirement Specifications (FRS).
- 4 years of working experience in Hadoop and its stack like HDFS, Hive, HBase, SPARK, SQOOP, ELASTIC SEARCH.
- Experience in importing and exporting data from relational database intoHadoopcluster using sqoop.
- Experienced in creating Hive, Pig and custom map reduce programs for analyzing data.
- Experience in validating and analyzing Hadooplog files.
- Experience writing Hive Queries for analyzing data in Hive warehouse usingHive Query Language (HQL).
- Experience in validating connectivity products that allow efficient exchange of data between core database engine and Hadoop ecosystem.
- Expert in writing complex SQL Queriesto check the integrity of data to perform database testing.
- Proficient in Data Extraction, Transforming and Loading (ETL) between Homogenous and Heterogeneous System using SQL tools .
- Have solid experience on database query tools such as SQL Navigator, Teradata SQL Assistant and SQL Plus.
- Experience in Creating and Updating Clustered and Non - Clustered Indexes to keep up the SQL Server Performance.
- Hands on experience in creating and managing fragmentation of Indexes to achieve better query performance.
- Strong in testing Stored Procedures, Functions, and packages utilizing PL/SQL.
- Written Hive UDF's in Python to perform to manipulate dates, string and execute the complex queries where default Hive functions failed to produce expected results. Excellent Understanding on data storage and retrieval techniques, ETL, and databases.
- Good experience with Talend open studio for designing ETL Jobs for Processing of data.
- Experience on UNIX commands and Shell Scripting
- Experienced in Functional or System testing, Unit Testing, Integration testing, Regression testing, UAT, GUI or Web-based Testing.
- Strong analytical, dynamic trouble-shooting and requirement traceability skills.
- Experienced in interacting with Clients, Business Analysts, leads, and UAT Users.
TECHNICAL SKILLS
Big Data: HDFS, MapReduce, Hive, HBase, Sqoop, Spark,, Kafka, Elastic Search.
ETL Tools: Informatica PC, Informatica IDQ, Informatic BDM.
Bug/Test Tracking System: HP ALM, JIRA, QTEST
Methodologies: Agile and Waterfall
Databases: Teradata, HBase, Hive QL, MS SQL 2003, 2007, 2014, Oracle, DB2.
Utilities: Putty, WinSCP, Eclipse, PyCharm.
Version Control Tool: GitHub
PROFESSIONAL EXPERIENCE
Confidential, Chicago, IL
Big Data/Hadoop Tester
Responsibilities:
- Come up with the requirements traceability matrix by participating in the requirements elicitation workshops and identifying requirements.
- Support the teams in testing with high quality deliverables, perform reviews in order to meet the defined quality SLAs.
- Work Collaboratively and Proactively with QA, Development, Business and other IT teams.
- Lead Integration/System/Performance/Automation and End to End testing based on integration and system test plans and support User Acceptance Testing.
- Work on the Agile scrum approach and all the testing approach.
- Work on design document, Mapping document to ensured proper transformation rules, any gaps are there to complete testing.
- Develop overall Test Strategy, lead testing of all impacted applications/infrastructure and post-production support activities.
- Work on Big data-HDFS, Hive, Pig, HBase, Kafka, Spark - Scala, PuTTY, JIRA, HP QC, QTest, TOAD, Tera Data SQL Assistant.
- Validate data from various layers from Ingestion to Ray layer and Gold layer, then Consumption.
- Lead and manage the offshore team - work allocation, issue support and other activities.
- Build Automation Framework using Hive Scripts to validate Source to Target Testing and generated reports. Published the report in the project Dash Board.
- Validate Audit Control Framework on Membership, claims data that are received from different systems to ensure the data landed properly on the data lake.
- Develop complex Pig, Hive, HBase, and shell scripts required for automation of scripts on HDFS.
- Bring value add to organization by building automation framework to validate big data.
- Data in the organization has been loaded into different databases depending on the Line of Business, to Performs Data Validation between the tables across different databases with complex HQL, PIG, SQL scripts.
- Perform defect management process - Logging all the bugs in JIRA and ALM for development and business for review.
- Perform the Data Quality checks, Audit checks on the source data and target.
- Run and monitor Hadoop Zena jobs and analyzing the data and generated reports to meet business requirements.
- Preparing the mock up/Synthetic data to test the possible positive and negative scenarios to meet the Business requirements.
- Validate data on Kafka topic - consumer and producer and compare the data from source.
- Build complex Hive SQL, modify and run the Sql in Big data - Hadoop Hive and validate data from various sources.
- Generate and analyze the metrics followed by fixing the issues and finally send the testing metrics to the Client. The metrics involves the following: Requirement coverage, Requirement Traceability, Defect Leakage by phase, Defect Density, Defect Removal Rate, Automation vs Manual test execution. Reporting tools are Grafana, Qtest and JIRA.
- Work on the multiple projects and involving with business key stakeholders on the various decisions.
Environment: Big Data, Hadoop, HiveQL, Sqoop, Spark, Scala, Kafka, Teradata, Informatica Windows 7, Qtest, UNIX, Putty, WinSCP.
Confidential
ETL Tester/Hadoop Tester
Responsibilities:
- Analyzed the Requirements from the client and developedTest cases based on functional requirements, general requirements and system specifications.
- Prepared test data for positive and negative test scenarios for Functional Testing as documented in the test plan.
- Used Bigdata validator to automate the testing process of the tables for large data, this was used to avoid manual testing.
- Prepared Test Cases and Test Plans for the mappings developed through the ETL tool from the requirements.
- Scheduling and automating jobs to be run in a batch process
- Identified various defects which helped the developer and team as the entire team was new to use Hadoop.
- Experienced in working in Agile Scrum and Waterfall SDLC methodology environments
- Tested complex SQL scripts for Teradata database for creating BI layer on DW for tableau reporting.
- Verified session logs to identify the errors occurred during the ETL execution.
- Created Test Cases, traceability matrix based on mapping document and requirements.
- Performed Teradata SQL Queries, creating Tables, and Views by following Teradata Best Practices
- Verified the logs to identify the errors occurred during the ETL execution.
- Reviewed the test cases written based on the Change Request document and Testing has been done based on Change Requests and Defect Requests.
- Tested theETLInformatica transformation and otherETLProcesses (DW Testing).
- Prepared Test Scenarios by creating Mock data based on the different test cases.
- Perform defect Tracking and reporting with strong emphasis on root-cause analysis to determine where and why defects are being introduced in the development process.
- Exclusively used test plan module to write and test lab module to execute the test cases in QTest.
- Tested several UNIX shell scripting for File validation and also PL/SQL programming
- Shared responsibility for administration of Hadoop, Hive and Pig Managed and reviewed Hadoop log files.
Environment: Big Data, Hadoop, HiveQL, Sqoop, Spark, Scala, Kafka, Teradata, Informatica Windows 7, Qtest, UNIX, Putty, PL/SQL, WinSCP.
Confidential, Southfield, MI
Hadoop Tester
Responsibilities:
- Prepared test cases, scripts based upon the business requirements documentation (BRD), use case documentations, and functional requirements specification (FRS).
- Performed Data Analysis for all incoming feeds toETL. Worked with Business Unit Managers for developing Mapping Document after Data Analysis.
- Created different objects: Stored Procedures, triggers script to populate data into different tables according to different parameters specified.
- CreatedHadoopjobs for processing and analyzing millions of records of data.
- Developed shell scripts to validate Hadoopdaemon services and reported accordingly to any warning or failure conditions.
- Validated the data load process for Hadoop using theHiveQL qurey’s.
- Error checking and testing of the ETL procedures and programs using Informatica session log.
- Focused on Data Quality issues/problems that include completeness, conformity, consistency, accuracy, duplicates, and integrity.
- Used Workflow Manager for Workflow and Session Management, Database Connection Management and Scheduling of jobs.
- Performed testing the ETL code and was also involved in Unit testing, System testing and integration testing of the project.
- Involved in validating the XML files coming from third party vendors.
- Validated the quality of data coming through vendors.
- Exclusively involved in execution of Autosys jobs, PL/SQL batch programs and responsible for reporting the defects to development team.
- Performed regression testing using QTP.
- Used TFS to store, schedule the test cases and report the Defects.
- Involved in preparing the Mock test data for both positive and negative scenarios.
- Involved in running SSIS ETL packages, and tested them according to requirements.
- Have performed Data Analysis on SSIS ETL packages, and worked on Data Validation as well. Involved in data integration of the project.
- Extensively performed backend testing on databases by writing complex SQL queries.
- DW was tested for the row counts and errors after each transaction loads.
- Coordinating with source system owners, day-to-dayETLprogress monitoring and maintenance of dailyInformaticabatch schedule run on a nightly basis.
- Defects were identified, provided documentation to the development team for debugging.
- Tested standard and Ad hoc reports and undergone data validation for the Cognos reports.
Environment: HDFS, Map Reduce, Hbase, Cassandra, Pig, Hive, Oracle 10g/11g, SQL, Data Analysis, InformaticaPower Center 9.1, TFS, QTP, SQL Server 2012, SSIS, SSRS, SSAS, Manual Testing, PL/SQL, Autosys, XML, XSLT, TOAD, Quality Center,, PuTTY, UNIX, Agile
Confidential, Bentonville, AR
ETL Tester
Responsibilities:
- Responsible in requirements gathering, analysis, design and development of any enhancements in the application.
- Involved in maintaining and updating the procedure for ETL process.
- Extensively involved in writing and managing SQL stored procedures, functions, triggers and packages to meet the business requirements and update the existing objects based on change requests.
- Created and maintained database objects like tables, views, materialized views, synonyms, procedures, functions and database triggers to meet the business requirements.
- Executed upload and load procedures to check the data feed is correctly getting loaded into the tables.
- Worked on ETL migration from DataStage to Talend Open Studio projects, handled various issues while code migration.
- Created Talend jobs to load data into various MySQL and PostgreSQL tables.
- Designing the ETL jobs using DataStage tool to load data from multiple source system to Teradata Database and Parallel jobs to load the data into the Target Schema.
- Used DataStage stages namely Datasets, Sort, Lookup, Peek, Standardization, Row Generator stages, Remove Duplicates, Filter, External Filter, Aggregator, Funnel, Modify, and Column Export in accomplishing the ETL coding.
- Testing the Jobs and preparing the Unit Test Cases & helped Business for User Acceptance test.
- Understanding the business logic to modify existing SQL Code and Performance Tuning, indexing, table partitioning.
- Modified the existing forms and reports and registered those in Application.
- Direct interaction with users and helped in the development of functional specifications.
- Wrote technical specifications for all the procedures developed in the module. Maintained log files during analysis and subsequently report any performance defects.
- Coordinate with the front-end design team to provide them with the necessary stored procedures, packages and the necessary data.
- Written UNIX shell scripts to automate processes like data load and sending notification mails.
Environment: SQL, Data Warehouse, Data Marts, ETL, Datastage, UNIX, Query Tuning, SQL Server Management Studio, SSDT.
Confidential, Jacksonville, FL
ETL Tester
Responsibilities:
- Experienced in validating the source data with the target data in data warehousing application and also in reports in Client/Server and Web Based environment.
- Involved in developing Test Cases, Test Plans, Test Execution, Defect Tracking, and Report Generation using Quality Center / HP ALM based on functional specifications.
- Involved in end-to-end defect management of assigned projects. Identified defects, assess root cause, and prepared detailed information for developers and business stakeholders.
- Experienced in Data Validation and Backend testing of databases to check the integrity of data.
- And also used extensively HQL Queries to analyze the HDFS data
- Used HP ALM for Test Management, Defect Management and save/manage the automation scripts created using QTP
- Experience in testing of Data Warehouse/ETL Applications developed in Informatica, Ab initio using SQL Server, Oracle, Hadoop, DB2, and UNIX and also have ability to evaluate ETL/BI specifications and processes.
- Experience in UNIX, RDBMS, Hadoop, HIVE (HQL), Oracle (PL/SQL), MS Access.
- Experienced in Black Box, White Box, Integration, Regression, Functional, Front End and Back End Testing.
- Involved in establishing automated Hadoop Integration testing system and implementing oozie workflow.
- Responsible for Analysis and Defect Tracking using HP Quality Center/ALM, Test Director, JIRA, IBM Clear Quest.
- Implemented Optimization techniques for better performance on the ETL side and also on the database side
- Experience with different file systems /databases like Oracle,HDFS Teradata, and MS SQL Server to extract and load data using sqoop.
Environment: HDFS, Map Reduce, Hbase, Cassandra, Pig, Hive, Oracle 10g, SQL, Data Analysis, InformaticaPower Center 9.1, Rational ClearCase, ClearQuest, TFS, Cognos, Soap UI,EDI
Confidential
SQL Developer
Responsibilities:
- Develop test and tune database code and information.
- Support product development and quality assurance teams.
- Design and maintain SQL scripts.
- Provide technical support and guidance to staff and clients.
- Generate reports and spreadsheets detailing database changes and performance.
- Designed Data Modeling, Design Specifications and to analyzeDependencies.
- Creatingindexeson tables to improve the performance by eliminating the full table scans and views for hiding the actual tables and to eliminate the complexity of the large queries.
Environment: SQL Server, Windows
