Big Data/hadoop Qa Test Lead Resume
Middle Town New, JerseY
SUMMARY
- 14+ years of experience in IT industry which comprising of 9+ years of experience in Quality Assurance and extensive experience in Data Science/Machine Learning, Big Data/Hadoop, Client/Server and of Web based applications and ETL Testing, Database and Mainframes applications.
- Currently perusing Master’s (MS) in Data Science from Eastern University
- Experienced with the Software Testing Life Cycle (SDLC) process for Waterfall and Agile development methodologies.
- Expertise knowledge ofHadoopHands on experience in working with Eco systems like Hive, Pig and Map Reduce, Sqoop, Oozie, Kafka, Hue and pySpark.
- Proficient in Functional testing, System testing, Regression Testing, Integration Testing, Database / ETL testing, User Acceptance Testing and Validation testing.
- Highly conversant in ORACLE/SQL queries and possess in DB2 and UNIX shell scripting, Python scripting and Big Data Hadoop.
- Expertise in Data Lake and Big Data Testing Expertise in Hadoop, Hive, pySpark, Hbase.
- Extensive hands - on experience in Web applications and delivering the product.
- Involved in various phases of a testing life cycle like Estimation, Requirement Analysis, Design, Test Plan Preparation, System testing and Test Reporting.
- Involved in All phases of Machine Learning Life cycle
- Strong technical knowledge of UNIX shell scripts as part of theHadoopand ETL process, automate the process of loading, pulling the data.
- Strong knowledge on Testing Machine Learning and Deep Neural Networks Models.
- Exceptional ability to maintain and build client relationships, able to comfortably converse with all facets in the client organization by working very closely with Customers, End users and powered users collect requirements and training or resolve any business process related issues.
- Worked closely with developers to design test criteria, test strategy, develop test plans, created and reviewed bug descriptions, functional requirements and design documents.
- Analysis, Design, Testing and Production Support using Data Warehouse, ETL, Informatica, Core Java and Mainframe applications.
- QA Lead experience with responsibilities in team management, onsite/offshore resource and task coordination, project management and issue resolution.
- Responsible for analyze big data and provide technical expertise and recommendations to improve current existing systems.
- Demonstrated ability to communicate and gather requirements, partner with Enterprise Architects, Business Users, Analysts and development teams to deliver rapid iterations of complex solutions
- Effective leadership quality with good skills in strategy, business development, client management and project management
- Ability to work effectively in cross-functional team environments and experience of providing training to business users.
- Excellent analytical, problem solving and communication skills, have been recognized by management and peer for producing high quality work
TECHNICAL SKILLS
Test Management Tools: HP Quality Centre/ALM, Microsoft Test Manager Test Professional 2013, JIRA
Operating Systems: Window NT/95/98/2000/XP/7 and UNIX/Linux
Database Systems: SQL / PL SQL Oracle7/8i/10g, TOAD
Languages: C, C++, Core Java, PL/SQL, Pig Latin, HiveQL, Linux & Python, Shell Scripting, Scala Machine Learning and Deep Learning Neural Networks
Mainframe Technologies: S/390 Mainframe, COBOL II, JCL, DB2, CICS, IMS DB and VSAM
Hadoop related Big Data Technologies: HDFS, YARN, MapReduce, Hive, Impala, Pig, Sqoop, Oozie, Spark, DistCp, MapR, Cloudera, Hortonworks, pyspark, Apache Hadoop, NoSQL(HBase), Vertica, Cornerstone, Cassandra, Confluence, Beeline, AWS, JIRA, Tableau and QlikView
Tools: & Utilities: RALLY, Tectia Client, Putty, Winscp, Autosys, Eclipse, Toad, Maven, FileZilla, SPUFI, AbendAid, Endevor, Remedy, QMF and File-Aid
Project Management: PRINCE2, ITIL V3.0, SAFE Agile Framework, Scrum
PROFESSIONAL EXPERIENCE
Confidential, Middle Town, New Jersey
Big Data/Hadoop QA Test Lead
Responsibilities:
- Involved in prioritising the user stories to maximise business value, taking into account risks, effort and dependencies involved
- Reviewed and re-prioritised the release backlog as per the change requests from product owner
- Analysed Business, Functional/Non-Functional Specifications, Detailed Design (DDL), High Level Design (HLD) and produced test plan documents
- Collaborated with the product owner and team in release planning and sprint planning meetings
- Construct and deploy both positive and negative test cases
- Discover, document and report bugs and performance issues.
- Executed Test cases as per the pre-defined acceptance criteria and raised defects
- Ensure the Map reduce jobs and Spark jobs are running Confidential peak performance
- Involved in prioritising the defects and issues with the product owner and the team
- Constituent Hadoop scripts like HiveQL and pyspark jobs are all robust
- Test and verify the machine learning applications according to requirements with appropriate datasets and data representation methods.
- Understanding business objectives and testing the models that help to achieve them, along with metrics to track their progress
- Prepare automated test scripts/test suites using python
- Manage and extend the automated test framework developed in Python using pytest
- Automation of test cases and assist in the maintenance of the internal Python-based testing libraries
- Algorithm evaluation using various techniques like K-Fold cross validation, Repeat Random Train Test Split method etc.,
- Performing the various Classification and Regression Metrics like AU-ROC, Confusion Matrix, Classification Report etc.,
- Analyzing the ML algorithms that could be used to solve a given problem and ranking them by their success probability
- Make sure for the High accuracy AI Deep Neural Network (DNN) Model which uses the optimal neurons in the hidden layer with parameter tunning.
- Ensure the pyspark and Hadoop jobs consumptions apt resources with the help of yarn logs
- Good experience in overall Quality Engineering process including Testing strategy, Test Plan and Test Metrics including all phasis of Test Lifecycle.
Environment: Hadoop, HDFS, HIVE, Pig, Sqoop, HBase, Spark, pyspark, Scala, Map Reduce, Cassandra, Oozie, Teradata, Vertica, MySql, GIT, Putty, Zookeeper, AWS, Linux Shell Scripting.
Confidential, New York
Big Data/Hadoop QA Test Lead
Responsibilities:
- Reviewed and re-prioritised the release backlog as per the change requests from product owner
- Analysed Business, Functional/Non-Functional Specifications, Detailed Design (DDL), High Level Design (HLD) and produced test plan documents
- Reviewed and re-prioritised the release backlog as per the change requests from product owner
- Executed Test cases as per the pre-defined acceptance criteria and raised defects
- Ensure the Map reduce jobs and Spark jobs are running Confidential peak performance
- Involved in prioritising the defects and issues with the product owner and the team
- Develop test cases and prioritize testing activities.
- Analyzed, designed and developed test cases for Big Data analytics platform using Hadoop, Hive, Spark and Pig Latin.
- Performed integration testing of Hadoop into traditional ETL, extraction, transformation, and loading of massive structured and unstructured data.
- Involved in prioritising the user stories to maximise business value, taking into account risks, effort and dependencies involved
- Validated aggregate data for reporting, dash boarding and ad-hoc analyses.
- Experience in scheduling control-M jobs as per the requirements and performing root cause analysis using the logs
- Prepare automated test scripts/test suites using python
- Manage and extend the automated test framework developed in Python
- Automation of test cases and assist in the maintenance of the internal Python-based testing libraries
- Experience in validating the bulk loading of external data source to Hadoop.
- Imported and exported data into HDFS and Hive.
- Exposure to CI / CD tools like Jenkins, Maven, Puppet etc.,
- Agile project methodology using JIRA software
- Developed Test strategy, test planning, test scenario and test data creation for the Enterprise Data Hub
- Produced regression test suite covering all the critical functionalities
- The enterprise data and advanced analytics strategy is implemented using quality assurance Best practices.
- Construct and deploy both positive and negative test cases
- Discover, document and report bugs and performance issues.
- Solved performance issues in Hive and Pig scripts with understanding of Joins, Group and aggregation and how does it translate to MapReduce jobs.
- Good experience in overall Quality Engineering process including Testing strategy, Test Plan and Test Metrics including all phasis of Test Lifecycle.
Environment: Hadoop, HDFS, HIVE, Pig, Sqoop, HBase, Spark, pyspark, Scala, Map Reduce, Cassandra, Oozie, Teradata, Vertica, MySql, GIT, Putty, Zookeeper, Linux Shell Scripting.
Confidential, Warren - New Jersey
Big Data/Hadoop QA Test Lead
Responsibilities:
- Develop test cases and prioritize testing activities.
- Analysed, designed and developed test cases for Big Data analytics platform using Hadoop, Hive, Pig Latin and pySpark
- Performed integration testing of Hadoop into traditional ETL, extraction, transformation, and loading of massive structured and unstructured data.
- Validated aggregate data for reporting, dash boarding and ad-hoc analyses.
- Experience in scheduling control-M jobs as per the requirements and performing root cause analysis using the logs
- Validated for transformation of transactions using Greenplum Postgres SQL as per requirement specification.
- Imported and exported data into HDFS and Hive.
- Participated in performance/stress testing activities, in collaboration with the development and operations teams, to ensure satisfactory performance of the system.
- Produced the test summary report Confidential the end of the project and reviewed with client
Environment: Hadoop, HDFS, HIVE, Pig, Sqoop, HBase, Spark, Scala, Map Reduce, Cassandra, Oozie, Teradata, MySql, SVN, Putty, Zookeeper, Linux Shell Scripting.
Confidential, Warren - New Jersey
Big Data/Hadoop QA Test Lead
Responsibilities:
- Monitored and performed integration testing of Hadoop packages for ingestion, transformation, and loading of data.
- Validated both Managed and External tables in Hive to verify optimize performance with the understanding of Partitions, Bucketing concepts
- Developed Oozie workflow for scheduling and orchestrating the automated testing scripts
- Created and worked Sqoop jobs with full/ incremental load to populate Hive External tables and to export data HDFS data to Oracle.
- Analyzed, designed and developed shell and hive scripts for validation of Big Data analytics platform- Benchmark Analytics.
- Attended daily scrum, sprint planning and weekly team status meetings to monitor and report the progress.
- Produced the test summary report Confidential the end of the project and reviewed with client
Environment: Hadoop, HDFS, HIVE, Pig, Sqoop, HBase, Map Reduce, Cassandra, Oozie, Teradata, MySql, SVN, Putty, Zookeeper, Linux Shell Scripting.
Confidential, New York
QA Tester
Responsibilities:
- Analyzing requirements and preparing test plan, effort estimation for upcoming projects.
- Documented the business requirements, developed test plans, test cases created for the Hadoop/Bigdata/NoSQL database backend testing and to test database functionality.
- Monitored the status of the Hadoop HDFS periodically and performed root-cause analysis on the issues identified.
- Validated the Map reduce, Pig, Hive Scripts by pulling the data from the Hadoop and validating it with the data in the files and reports.
- Assisted in gathering the business requirements, Hadoop Bigdata ETL Analysis, Bigdata ETL test and design of the flow and the logic for the Data warehouse project
- Backend testing of the DB by writing scripts to test the integrity of the application NoSQL/Hadoop databases.
- Closely went through and worked on all the stages of SDLC for this project and designed and executed Functional, Integration, Regression, System (End to End) and Backend (Hadoop Database).
- Solved performance issues in Hive and Pig scripts with understanding of Joins, Group and aggregation and how does it translate to MapReduce jobs.
Environment: Hadoop, HDFS, HIVE, Pig, Sqoop, HBase, Map Reduce, Cassandra, Oozie, MySql, SVN, Putty, Zookeeper, Linux Shell Scripting.
Confidential - Dearborn, MI
QA Tester
Responsibilities:
- Monitored the status of the Hadoop HDFS periodically and performed root-cause analysis on the issues identified.Performed Data Validation testing by using simple to complex SQL queries which involves fields with calculations / functions
- Handled UNIX shell scripts and commands to check the process logs and data loading activities.
- Assisted in gathering the business requirements, Hadoop Bigdata ETL Analysis, Bigdata ETL test and design of the flow and the logic for the Data warehouse projectPerformed Smoke Testing, Functional testing, Regression testing, and also System integration testing on need basis.
- Analyzing requirements and preparing test plan, effort estimation for upcoming projects.
- Preparation and Review of Test Scenarios and Test Cases.
- Test Execution and reporting of issues/defects in QC
Environment: Hadoop, HDFS, Map Reduce, HIVE, Pig, Sqoop, HBase, Oozie, My Sql, SVN, Putty, Zookeeper, Ganglia, Linux Shell Scripting, COBOL II, DB2, IMS DB, VSAM, JCL, CA-7
Confidential, NYC, NY
Responsibilities:
- Analyzing requirements and preparing test plan, effort estimation for upcoming projects.
- Preparation and Review of Test Scenarios and Test Cases.
- Created and Processed transactions to create test data.
- As part of this assignment, performed Functional, Regression testing and End to End testing.
- ETL Testing with data warehousing tool Informatica.
- Handled oracle database (PL/SQL developer)test execution as team player
- Handled UNIX shell scripts to check the process logs.
- Interacted with Clients on a frequent basis to get the inputs and status reporting.
- Lead a team working in multiple locations.
- Communicate deliverables status to user/stakeholders, client and drive periodic review meetings.
- On time completion of tasks and the project per quality goals.
Environment: Informatica PowerCenter 9.1, Oracle, SQL, PL/SQL, Remedy, SQL Developer, Flat Files, FileZilla, Shell scripting, COBOL II, DB2, IMS DB, VSAM, JCL, CA-7
Confidential - Michigan, MI
System Analyst
Responsibilities:
- Involved in source system Analysis and business requirement gathering with users
- Worked closely with the team responsible for gathering the reporting needs as well as the ensuring that the sourced data is not currently available in the existing data warehouse
- Worked closely with the data modellers to come up with the data model and ensure that it confirms to dimensional modeling reporting needs
- Performed Smoke Testing, Functional testing, Regression testing, and also System integration testing on need basis.
- Providing direction and guidance to a team of developers, including allocation and management of workload by conducting status calls and code reviews.
Environment: Informatica PowerCenter 9.1, COBOL, JCL, VSAM, CICS, DB2, IMS-DB, Easytrive, ENDEVOR, SPUFI, FILE-AID, CA7.
Confidential, Detroit, MI
Technical Lead
Responsibilities:
- Worked closely with the team responsible for gathering the reporting needs as well as the ensuring that the sourced data is not currently available in the existing data warehouse.
- Responsible for end to end verification of requirements gathered and the functional specifications and come up with technical design document and Source to Target mappings documents.
- Development and review of mappings involving extracting data from Flat Files, Oracle, sources to Oracle database.
- Involved in Development team reviews like Review of code, unit test cases and results, System and Integration test cases and results and promote the CL’s to Endevor Model region.
- Involved in performance tuning the ETL processes, testing of stored procedures and functions, testing of Informatica sessions, batches and the target Data.
- Involved in Development team reviews like Review of code, unit test cases and results, System and Integration test cases, results and promote the CI’s to Endevor Model region.
Environment: Informatica, COBOL, JCL, VSAM, CICS, DB2, IMS-DB, Easytrive, ENDEVOR, SPUFI, FILE-AID, CA7
Confidential, Charlotte, NC
Business Associate
Responsibilities:
- Interacting with client to define business requirements and scope of the project.
- Creation of System Test Cases, Unit Test Cases and review document, Migration Documents
- Maintained, developed and fixed bugs for applications.
- Solid background in Object-Oriented analysis and design.
- Executing test cases and fixing bugs through unit testing.
- Generating daily progress reports
- Monitoring of Daily Production Jobs, Production support of Application.
- Working with clients on requirements based on their business needs.
Environment: COBOL, JCL, VSAM, DB2, Java, JSP, Servlets, XML, Rational Rose, Web Services, Windows XP, LINUX