SUMMARY:

• Total 9.5 years of Experience which includes:
• 2.5 years of Hadoop Development in Big Data domain
• 4 years of Automation testing in Embedded domain
• 3 years of Manual testing in Embedded domain
• Developed Pig and Hive scripts to process STB/Router log data
• Developed Pig and Hive UDFs in Python to do data transformations using Regular expressions
• Used Python and implemented machine Learning algorithms in Python to find patterns in the logs
• Developed Pig, Hive and Python scripts for validating the MapReduce algorithms developed for data transformation and data analysis
• Developed Shell scripts to enable running hadoop commands using parameters
• Used Flume for Data ingestion
• Knowledge of Scala, SparkSQL, Spark Streaming and Spark MLlib
• Experience in using Oozie workflow manager
• Knowledge of NoSQL databases like HBase and Sqoop
• Knowledge of Core Java and developing Mapreduce programming in Java
• In depth understanding of Hadoop cluster Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MRv1 and MRv2 (YARN).
• Capable of processing large sets of structured, semi - structured and unstructured data
• Understanding of different storage formats - JSON, Avro, Parquet, ORC, Sequence files etc.
• Knowledge of Machine Learning algorithms
• Knowledge of ETL concepts
• Understanding of Web services - SOA, XML, SOAP, REST etc.
• Experience of working in Scrum Agile Methodology
• Created Automation Testing Algorithms and written Scripts using Python and VBScript
• Preparation and review of - Test Cases, Test Strategy, Test Plan and Test Reports
• Involved in different kinds of Testing - Sanity, System, Functional, Regression, Stress, Performance and Exploratory
• Well versed with different Test Case Design Techniques- OATS, Decision table, State based testing, Cause - effect graphing

AREAS OF EXPERTISE:

Operating Systems: Linux Family (RHEL, Ubuntu), Windows Family

Hadoop Ecosystem: Hortonworks Hadoop distribution, HDFS, MapReduce, Pig, Hive, HBase, Sqoop, OozieFlume, Spark, Scala, Kafka, Splunk

Languages: Python, Core JAVA, VB Script, Shell Script, Perl

Automation Tools: STB Automation Tools developed in .NET framework

Defect Management Tools: JIRA, Clear Quest, Clarify, Change Synergy

Test Management Tools: HP ALM, Quality Central, TestLink

Databases: MySQL

PROFESSIONAL EXPERIENCE:

Confidential, Basking Ridge, NJ

Tools & Environment: Hortonworks, Flume, Pig, Python, Hive, HDFS, Spark Splunk

Hadoop Developer

Responsibilities:

• Collected high volume of STB logs/Router Logs using Flume and staged data in HDFS for further analysis
• Developed Pig Latin scripts to analyse the STB logs/Router logs
• Processed the semi structured data in JSON format
• Developed Pig UDFs in Python to implement complex regular expressions and for creating ngrams.
• Created Hive tables, partitions and Used HCatalog to access Hive through Pig
• Developed Shell scripts for running Pig and Hive scripts with parameters and for Multi-level partitioning in Hive
• Configured Oozie workflow engine to run multiple Hive and pig jobs
• Implemented DBSCAN Machine learning algorithm using Python ( pylab ) and cluster the logs to identify patterns
• Providing log analysis information to the developers which helps them to fix the Defects
• Used Splunk to access the access the customer tickets

Confidential

Tools & Environment: Hortonworks Hadoop, MapReduce, Java, HDFS, Hive, Pig, Sqoop, HP ALM, JIRA

Hadoop Tester

Responsibilities:

• Responsible for validating the Hadoop MapReduce algorithms developed in Java before releasing them in production.
• Validated the migration from MRv1 to MRv2
• Developed Pig scripts to validate the logic implemented.
• Validated the data imported to Hive using Sqoop
• Worked in Scrum Agile model of Software Development

Confidential

Tools & Environment: ThinkAnalytics Recommendation Engine, HTML 5 EPG, Nagra MDG server, Python, SmarTAM, SQL

Data Engineer

Responsibilities:

• Testing the integration of Recommendation Engine with Starhub STB
• Validating the Collaborative filtering algorithms which recommend EPG and VOD content to the customers depending on profile, previous usage behavior and user ratings
• Testing the Television Audience measurement Technology which collects the real time data from the customers
• Testing the Business rules Engine for marketing, package configuration and customer segmentations based on the data collected from the customers
• Created SQL queries and Python automation scripts in testing the above algorithms

Confidential

Tools & Environment: IEX, VB Script, Java EPG, MySQL, HTML, Shell Script, Jenkins, Ubuntu

Tech Lead - QA

Responsibilities:

• Devised Automation Algorithm to automate all Java based EPG Navigation & Presentation Test cases using one Script and one Excel Sheet. Implemented this strategy using IEX Tool with VBScript programming
• Used Bitmap and OCR checking for automating Presentation Test cases
• Used Shell Script for MySQL database backup
• Received Quarterly Milestone Award for this Automation Strategy
• Worked in Scrum Agile model of Software Development

Confidential

Tools & Environment: Python, MV Linux, boa web server, RHEL, Shell Script, H.264, MPEG 4, JPEG, RTSP, UPnP, FTP, HTTP, SMTP, SNTP, Clear Quest

Lead Product Engineer

Responsibilities:

• Lead the System Testing Team of 2 members and involved in preparation of Test Strategy and Test Plan
• Involved in writing Test cases and testing features like H.264, MPEG4, and JPEG codec streaming, Video Analytics (Motion Detection, Face Recognition), Video Processing, Digital I/O, UPnP, Web Server, RTSP and DVR Recording
• Automated IPNC Testing using Python, making use of built in Web server, and reduced Test cycle time by 50%
• Used Regular Expressions in Python scripting in the process of Automation

Confidential

Tools & Environment: DAC 6000 Headend, Seachange VOD Server, Digicipher II, Mercury EPG, Clarify

Senior Software Engineer

Responsibilities:

• Involved in Confidential STB Testing Setup and DAC 600 Headend setup at Confidential Lab, Bangalore
• Involved in System Testing of different SD/HD Set Top Boxes in a HMA (Home Media Architecture)
• Prepared Test cases and Tested features like WH DVR, Closed Captioning, IPPV, Media Manager