SUMMARY

• A Qualified IT Professional wif 7+ years of experience including 3+ years of experience as a Hadoop developer
• Worked experience of various phases of SDLC such as Requirement Analysis, Design, Code Construction and Test
• Having 3+ years of experience as a Hadoop Developer/Administrator
• Having Experience on HDFS, MapReduce, Hive, Pig, Sqoop, Flume, Oozie, HBase, Impala, Zookeeper, Hue
• Good Knowledge on Hadoop Eco System components, Cluster architecture and monitoring teh cluster.
• Having basic knowledge about real - time processing tools Storm, Spark
• Experienced in analyzing data using HiveQL, Pig Latin, and custom MapReduce programs in Java
• Having Experience on monitoring tools Ganglia, Cloudera Manager, and Ambari
• Experienced in importing and exporting data using Sqoop from HDFS to Relational Database Systems, Teradata and vice versa
• Experienced in analyzing and recommending Big Data solutions.
• Experienced in NOSQL Databases concepts and Hbase and MongoDB.
• Experienced in providing Business Intelligence Solutions, data Extraction Transform and Loading (ETL) using SQL Server Integration Services (SSIS)
• Real time experience on Production deployment, code fixes.
• Basic knowledge on Machine Learning and Predictive Analysis.
• Worked on Talend data visualization tools
• Wrote custom UDF’s for extending Hive and Pig core functionality
• Worked on Postgres, MySQL, Oracle 10g
• Development experience wif IDE’s Eclipse and NetBeans
• Having experience wif Schema design using like HBase, Cassandra
• Passionate about working on teh most cutting-edge Big Data technologies.
• Ability to adapt to evolving technology, strong sense of responsibility and accomplishment
• Willing to update my knowledge and learn new skills according to business requirements

TECHNICAL SKILLS

Hadoop Technologies: HDFS,MapReduce,Hive,Impala,Pig,Sqoop,Flume,Oozie,Zookeeper,Ambari,Hue,Spark,Strom,Talend,Ganglia

Operating System: Windows, Unix, Linux

Languages: Java, J2EE, SQL, PL/SQL, Shell Script

Project Management / Tools: MS Project, MS Office, TFS, HP Quality Center Tool

ETL Technologies: Microsoft SQL Server Integration Services (SSIS 2008 Microsoft SQL Server Integration Services (SSIS 2008)

Front - End: HTML, JSTL, DHTML, JavaScript, CSS, XML, XSL, XSLT

Databases: MySQL, Oracle 11g/10g/9i, SQL Server

NoSQL Databases: HBase, Cassandra

File System: HDFS

Reporting Tools: Jasper Reports, Tableau

IDE Tools: Eclipse, NetBeans

Application Server: Apache Tomcat, Web Logic

PROFESSIONAL EXPERIENCE

Confidential - Foster City, CA

Hadoop Developer

Responsibilities:

• Responsible for building scalable distributed data solutions using Hadoop.
• Written multiple Map Reduce programs in Java for Data Analysis.
• Wrote Map Reduce job using Pig Latin and Java API.
• Performed performance tuning and troubleshooting of Map Reduce jobs by analyzing and reviewing Hadoop log files.
• Worked wif HiveQL on big data of logs to perform a trend analysis of user behavior on various online modules.
• Passionate about working on teh most cutting-edge Big Data technologies.
• Developed Pig scripts for analyzing large data sets in teh HDFS.
• Collected teh logs from teh physical machines and teh OpenStack controller and integrated into HDFS using Flume.
• Designed and presented plan for POC on Impala.
• Experienced in migrating HiveQL into Impala to minimize query response time.
• Knowledge on handling Hive queries using Spark SQL dat integrates wif Spark environment.
• Implemented Avro and parquet data formats for Apache Hive computations to handle custom business requirements.
• Responsible for creating Hive tables, loading teh structured data resulted from MapReduce jobs into teh tables and writing Hive queries to further analyze teh logs to identify issues and behavioral patterns.
• Worked on Sequence files, RC files, Map side joins, bucketing, partitioning for Hive performance enhancement and storage improvement.
• Imported data from mainframe dataset to HDFS using Sqoop. Also handled importing of data from various data sources (i.e. Oracle, DB2, Cassandra, and MongoDB) to Hadoop, performed transformations using Hive, MapReduce.
• Implemented Daily Cron jobs dat automate parallel tasks of loading teh data into HDFS using Oozie coordinator jobs.
• Responsible for performing extensive data validation using Hive
• Sqoop jobs, PIG and Hive scripts were created for data ingestion from relational databases to compare wif historical data.
• Involved in loading data from Teradata database into HDFS using Sqoop queries.
• Involved in submitting and tracking MapReduce jobs using Job Tracker.
• Involved in creating Oozie workflow and Coordinator jobs to kick off teh jobs on time for data availability.
• Used Pig as ETL tool to do transformations, event joins, filter and some pre-aggregations
• Used Visualization tools such as Power view for excel, Tableau for visualizing and generating reports.
• Exported data to Tableau and excel wif Power view for presentation and refining
• Implemented business logic by writing Pig UDFs in Java and used various UDFs from Piggybanks and other sources
• Implemented Hive Generic UDF's to implement business logic.
• Implemented test scripts to support test driven development and continuous integration.
• Involved in story-driven agile development methodology and actively participated in daily scrum meetings.

Environment: Apache Hadoop, Map Reduce, HDFS, Pig, Hive, Sqoop, Flume, Oozie, Java, Linux, Maven, Teradata, Zookeeper, Tableau.

Confidential, Dearborn, MI

Hadoop Developer

Responsibilities:

• Worked on writing transformer/mapping Map-Reduce pipelines using Java.
• Hadoop is a big data platform dat is used to store and analyze teh data.
• Involved in creating Hive Tables, loading wif data and writing Hive queries which will invoke and run MapReduce jobs in teh backend.
• Designed and implemented Incremental Imports into Hive tables.
• Worked in Loading and transforming large sets of structured, semi structured and unstructured data
• Involved in collecting, aggregating and moving data from servers to HDFS using Apache Flume
• Written Hive jobs to parse teh logs and structure them in tabular format to facilitate TEMPeffective querying on teh log data
• Experienced in managing and reviewing teh Hadoop log files.
• Migrated ETL jobs to Pig scripts do Transformations, even joins and some pre-aggregations before storing teh data onto HDFS.
• Implemented teh workflows using Apache Oozie framework to automate tasks
• Worked wif Avro Data Serialization system to work wif JSON data formats.
• Worked on different file formats like Sequence files, XML files and Map files using MapReduce Programs.
• Involved in Unit testing and delivered Unit test plans and results documents using Junit and MRUnit.
• Developed scripts and automated data management from end to end and sync up between all teh clusters.
• Involved in Setup and benchmark of Hadoop /HBase clusters for internal use.
• Created and maintained Technical documentation for launching Hadoop Clusters and for executing Pig Scripts.

Environment: Hadoop, HDFS, Map Reduce, Sqoop, Oozie, Pig, Hive, Flume, LINUX, Java, Eclipse, Cassandra.

Confidential, Charlotte, NC

Hadoop Admin/ Developer

Responsibilities:

• Loaded customer data such as service installations, technical halp line calls and interaction from teh dish network web site in to HDFS using Flume.
• Implemented 26 node CDH4 Hadoop cluster on Red hat Linux using Cloudera Manager.
• Import data from Teradata, Oracle to HDFS using Sqoop.
• Optimized MapReduce Jobs to use HDFS efficiently by using various compression mechanisms.
• Handled importing of data from various data sources, performed transformations using Simple to complex MapReduce jobs, and loaded data into HBase.
• Analyzed teh data by performing Hive queries and running Pig scripts to study customer behavior.
• Continuous monitoring and managing teh Hadoop cluster using Cloudera Manager.
• Setup Flume agent on teh application servers to capture teh log data.
• Setup flume agent on teh web servers to capture teh server logs.
• Used Avro sink and sources to move teh file into HDFS.
• Analysis of teh top errors occurring post an Integrated release using Hive and Pig.
• Used Sqoop to move teh analyzed data to teh MySQL DB for report generation.
• Setup Amazon web services (AWS) to check whether Hadoop is a feasible solution or not.
• Setup Hadoop cluster using EC2 (Elastic MapReduce) on managed Hadoop Frame Work.
• Used Maven extensively for building MapReduce jar files and deployed it to Amazon Web Services (AWS) using EC2 virtual Servers in teh cloud.
• Used S3 Bucket to store teh jar’s, input datasets and used Dynamo DB to store teh processed output from teh input data sets.
• Setup MongoDB to store teh ever growing application config entries in JSON format.
• Use MongoDB as a contingency DB for teh current Oracle clusters.
• Setup a MongoDB cluster wif 12 shards.
• Setup Hadoop cluster using CDH5.0
• Used Sqoop to and mongo Dump to move teh data between MongoDB and HDFS.

Environment: CDH4/5, Cloudera Manager, MapReduce, HDFS, Hive, Pig, HBase, Flume, MySQL, Sqoop, Oozie, AWS, MongoDB.

Confidential, Scottsdale AZ

SQL Developer

Responsibilities:

• Defined and designed teh data sourcing and data flows, data quality analysis.
• Identified teh data source and defined them to build teh data source views.
• Wrote Complex Stored Procedures, Triggers, Views and Queries. Created indexes, Constraints and rules on database objects.
• Developed and deployed SSIS packages to load data from OLTP to Data Warehouse system.
• Designed ETL packages dealing wif different data sources (SQL Server, CSV, Flat Files etc.) and loaded teh data into target data destination by performing different kinds of transformations using SQL Server Integration Services (SSIS).
• Used SQL Server Agent to automate teh SSIS 2005 package execution.
• Developed stored procedures to transform teh data from enterprise data warehouse to SQL server and load data to fact tables.
• Transferred data from various sources like MS Excel, MS Access, and SQL Server using SSIS 2005 and then created reports using dis data using SSRS.
• Configured and maintained Report Manager and Report Server for SSRS.
• Identified and added teh report parameters and created teh reports based on teh requirements using SSRS 2005.
• Tested and managed teh SSIS 2005 and SSIS 2005 packages and was responsible for its security.

Environment: MS SQL Server 2005, MS SQL Server Reporting Services, SQL, MS Excel, MS word, Visual Studio 2005.