IT professional with 6 years of diversified experience with extensive knowledge and background in Software Development lifecycle- Analysis, Design, Development, Debugging and Deploying various software applications. More than 2 years hands on experience in BigData and Hadoop Ecosystem including MapReduce, Pig, Hive, and Sqoop.

• Over 2 years of professional work experience in Hadoop Cloudera distribution CDH3, 4 and 5 on clusters of 65 nodes.
• Extensive experience in both MapReduce MRv1 and MapReduce MRv2 YARN
• Extensive experience in HDFS, PIG, Hive, Sqoop, Flume, Oozie, Zookeeper and HBase
• Experience with Cloudera CDH3, CDH4 and CDH5 distributions
• Extensive experience with ETL and Query big data tools like Pig Latin and Hive QL
• Hands on experience in big data ingestion tools like Flume and Sqoop
• Experience with Sequence files, AVRO and HAR file formats and compression.
• Experience in tuning and troubleshooting performance issues in Hadoop cluster.
• Hands on NoSQL database experience with HBase .
• Experience in designing, sizing and configuring Hadoop environments
• Expertise with managing and reviewing Hadoop log files.
• Knowledge of BI platform such as Tableau.
• Hands on Elastic Search, Kibana4.
• Experience in data masking, encryption using Dataguise.
• Background with traditional databases such as Oracle, SQL Server, MySQL.
• Good understanding of ETL processes.
• Well versed in Core Java.
• Hands on experience on IDE tools like Eclipse, NetBeans, Visual Studio.
• Experience in collecting business requirements, writing functional requirements and test case documents Creating technical design documents with UML Use Cases, Class, and Sequence and Collaboration diagrams.
• Well versed in Object Oriented Programming and Software Development Life Cycle from project definition to post-deployment
• Refined planning and organizational skills that balance work, team support and ad-hoc responsibilities in a timely and professional manner.
• An individual with excellent interpersonal and communication skills, strong business acumen, creative problem solving skills, technical competency, team-player spirit and leadership skills.
• Ability to effectively communicate with all levels of organization such as technical, management and customers.

Confidential

Hadoop Developer

Responsibilities:

• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
• Used Sqoop extensively to ingest data from various source systems into HDFS.
• Written Hive queries for data analysis to meet the business requirements
• Familiarized with job scheduling using Fair Scheduler so that CPU time is well distributed amongst all the jobs.
• Involved in the regular Hadoop Cluster maintenance such as patching security holes and updating system packages.
• Executed queries using Hive and developed Map-Reduce jobs to analyze data.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Developed Hive queries for the analysts.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig.

Environment: Eclipse, Oracle 10g, Hadoop, MapReduce, Hive, Linux, MapReduce, HDFS, Hive, MapR, SQL, Toad 9.6.

Confidential

Hadoop Consultant

One of the challenges retailers and e-commerce provider's face today is maintaining customer loyalty while reducing email opt-outs. Macy's collects and analyzes large amounts of data from our customers 24 7 from several data points websites, mobile apps, Macy's Credit card, social media and coupons redemption. Data from these data points could be structured and unstructured in few cases. All these data is collected, aggregated and analyzed in the Hadoop cluster to find shopping patterns make cross sell, up sell business decisions and devise targeted marketing strategies.

Responsibilities:

• Worked on a live 65 nodes Hadoop cluster running CDH4.4
• Extensive experience in writing Pig scripts to transform raw data from several data sources in to forming baseline data.
• Developed Hive scripts for end user / analyst requirements for adhoc analysis
• Very good understanding of Partitions, Bucketing concepts in Hive and designed both Managed and External tables in Hive for optimized performance
• Solved performance issues in Hive and Pig scripts with understanding of Joins, Group and aggregation and how does it translate to MapReduce jobs.
• Worked in tuning Hive and Pig scripts to improve performance.
• Experience in using Sequence files, AVRO and HAR file formats.
• Extracted the data from Teradata into HDFS using Sqoop.
• Created Sqoop job with incremental load to populate Hive External tables.
• Developed Oozie workflow for scheduling and orchestrating the ETL process
• Very good experience with both MapReduce 1 Job Tracker and MapReduce 2 YARN
• Continuous monitoring and managing the Hadoop cluster through Cloudera Manager.
• Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files.
• Configured Hadoop system files to accommodate new sources of data and updated the existing configuration Hadoop cluster
• Involved in loading data from UNIX file system to HDFS.
• Involved in gathering business requirements and prepared detailed specifications that follow project guidelines required to develop written programs.
• Actively participating in the code reviews, meetings and solving any technical issues.

Environment: Eclipse, Oracle 10g, Hadoop, MapReduce, Hive, HBase, Oozie, Linux, MapReduce, HDFS, Hive, CDH, SQL, Toad 9.6.

Confidential

BigData Consultant

The purpose of the project is to improve customer's shopping experience with Sears. Collect click stream data from Sears websites and mobile apps and analyze the shopping patterns with these application and customize customer facing applications to make it user friendly for the customer in reaching the products in fewer clicks and also personalizing the shopping experience. Hadoop is used to collect and store data from various data points and perform various analyses on these data using MapReduce jobs, Pig and Hive Scripts. Aggregated results are then exported over to downstream RDBMS for Business Intelligence reporting.

Responsibilities:

• Worked on a live Hadoop production CDH3 cluster with 35 nodes
• Worked with highly unstructured and semi structured data of 25 TB in size
• Good experience in benchmarking Hadoop cluster.
• Implemented Flume Multiplexing to steam data from upstream pipes in to HDFS
• Used Sqoop to import data from DB2 system in to HDFS
• Worked on custom Map Reduce programs using Java
• Designed and developed PIG data transformation scripts to work against unstructured data from various data points and created a base line.
• Worked on creating and optimizing Hive scripts for data analysts based on the requirements.
• Very good experience in working with Sequence files and compressed file formats.
• Worked with performance issues and tuning the Pig and Hive scripts.
• Good experience in troubleshooting performance issues and tuning Hadoop cluster.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
• Good experience in setting up and configure clusters in AWS.
• Worked with the infrastructure and the admin teams to set up monitoring probes to track the health of the nodes
• Created and maintained Technical documentation for launching Hadoop Clusters and for executing Hive queries and Pig Scripts.

Environment: Eclipse, Oracle 10g, Hadoop, Hive, HBase, Oozie, Linux, MapReduce, HDFS, Hive, CDH, SQL, Toad 9.6.

Confidential

Description:

Item Authority Audit Tool delivers an Item Authority Quality Audit that measures the quality of the customer experience in terms of product uniqueness. This audit will focus on the Search Result/Browse experience where defects in Item Authority have the most customer impact. Using this, a random sampling of terms will be taken and each term's search results will be manually audited for item authority defects. Thus you will enable both an absolute quality metric and customer impact metrics in the form of spurious traffic, search impressions and detail page hits to non-authoritative ASINs. The goal of this tool is efficiently provide an accurate data page quality.

Roles and Responsibilities:

• Understanding the Functional Requirements and Specifications.
• Creation of Test Cases and execution of Test cases.
• Experience in reviewing the Test Case documents.
• Performed Functional, Integration, System Testing.
• Identifying the test cases for automation.
• Responsible for sending daily and weekly status report.

Confidential

ETL/SQL Test Analyst

This project is to support an initiative to implement a revenue data warehouse with an integration of nine billing systems used across multiple countries to meet the management accounting system needs for profitability analysis and FP A needs for financial reporting.

Responsibilities:

• Owned Data integration of one of the complex billing systems: Contract Management System, which deals with the contracts of Commercial customers for Equifax Canada. Proposed and implemented a Conformed Facts technique, a technique where source rules are phased in earlier in the ETL layers.
• Gathered data inputs for the source system depending on the business requirement. Responsible for preparation of ETL Strategy Document / Design Mapping Specification Documents / Test Case Specifications for the Source System dealt. Designed and implemented logical and physical data-model using Erwin.
• Analyzed Functional Design Document and prepared necessary test setup. Developed and tested source to target parallel jobs with data validations, SCD Type 2 jobs for all dimensions in the data warehouse and handled incremental load jobs.
• Was involved in the data testing on Oracle Database writing SQL queries and check whether right data from the source system has been populated using SQL Developer.
• Worked on data Re-conciliation in Source System with the General Ledger by writing complex SQL queries. Performed historical data load using SQL Loader scripts.
• Populated test data and tested whether correct data is being populated into the database while applying business rules.
• Developed and tested Audit Scripts in Unix Shell for calculating the no. records received in the data file matches with the no. records sent, Control Scripts for passing the parameters, running the parallel jobs developed in the DS Environment Scripts. Shell script for calculating the rejected records threshold.
• Tested the reports using Microstrategy ETL tool and observed the results of the report with correct data population.
• Test Data preparation, written and executed test cases for Functional testing, Executing test cases.
• Documented defects and verified fixes. Reviewing test cases and scenarios. Reporting the bugs.
• Effectively worked with AR AP systems for the reconciliation of the Revenue Amounts with the revenue Data warehouse.
• Used Quality Center as a centralized store for organizing and managing the testing process.

Environment: Oracle 10g Database, SQL, Informatica, Microstrategy, UNIX, Quality Center

Confidential

Oracle E-Business Suite

Responsibilities:

• Design and Developing of workflow procedures.
• Has done many migrations of project's application interfaces.
• Was responsible for handling the Billing application and related Change requests.
• Implemented PL and SQL queries, triggers and Stored Procedures as per the design and development related requirements of the project.
• Has done end to end testing of the applications and Error casual analysis.
• Has done proper documentation of the application I've developed, for better future understanding.
• Prepare project-phase plans and implementation schedule, plan and conduct meeting and workshops relative to the project plans, and communicate day to day project status to all levels of management.