SUMMARY OF EXPERIENCE:

• 8 Years of experience in IT industry comprising of extensive work experience in Big Data technologies and data analytical solutions.
• Experienced Hadoop Developer with good knowledge of Hadoop framework, HDFS (Hadoop Distributed file system) and Parallel Processing implementation, Hadoop Ecosystem, Map Reduce, Hive, Pig, Hbase, Sqoop, Hue, Oozie, and Impala.
• Excellent understanding/knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map - Reduce programming paradigm.
• Experienced in handling different file formats like a Text file, Avro data files, Sequence files, and XML and JSON files.
• Wrote custom Map-Reduce programs for data Processing and UDFs for both Hive and Pig in Java. Extensively worked on MRV1 and MRV2 Hadoop architectures.
• Extensive experience in working with structured data using Hive QL, writing custom UDF's and experienced in optimizing Hive Queries.
• Experience in importing and exporting data using Sqoop form HDFS to Relational Database.
• Experience in Apache Flume for collecting, aggregating and moving huge chunks of data from various sources such as web server, telnet sources etc.
• Experience with Oozie Workflow Engine in running workflow jobs.
• Excellent understanding and knowledge of NoSQL databases like Hbase and Cassandra.
• Great team player and quick learner with effective communicating, motivating, and organizational skills combined with attention to details and business improvements.
• Excellent knowledge about Cloudera, Hadoop distribution components, and their custom packages.
• Complete Knowledge in installing, configuring and using Hadoop Ecosystem components.
• Performed Data transfer between HDFS and their Relational Database Systems (MySQL, SQLServer, Oracle and DB2) using Sqoop.
• Developed various UDFs in Map-Reduce, Pig, and Hive.
• Experienced in the SQL and NoSQL Databases like MySQL, MS SQL, Mongo DB, Hbase and Cassandra.
• Experience in working with an onsite-offshore model.
• Worked in IT, banking and refinery Information technology domains.
• Proficient in Hive Query language and experienced in hive performance optimization using Partitioning, Dynamic Partitioning and bucketing concepts.
• Specialist in creating Hive tables, lading with data & writing hive queries.
• Good experience in Linux, Mac S environment.
• Used various development tools like Eclipse.
• Extensively worked on the flowing components of Hadoop Ecosystem Map Reduce, Hive, Pig, Sqoop, Impala, Flume, Oozie, Hbase, and YARN.

TECHNICAL SKILLS:

Hadoop ECO Systems: Hadoop, MapReduce, HDFS, HBase, Hive, Pig, Sqoop, ZooKeeper, Flume, Impala, Hue, andOozie.

NOSQL/Data Bases: Hbase, Cassandra, MongoDB, MS SQL Server, MY SQL, MS access

Languages: HTML, Java script, Java, SQL.

Operating Systems: Windows XP/Vista, Mac OS, UNIX, LINUX. Worked on Eclipse, Maven, and NetBeans before.

Business Intelligence: Tools SAP Lumira.

ETL Tools: SAP Data server, SAS Data Management, and QlikView.

Methodologies: Agile, Scrum.

PROFESSIONAL EXPERIENCE

Hadoop Developer

Confidential, Philadelphia, PA

Responsibilities:

• Helped in analyzing the requirement to setup a cluster.
• Worked on analyzing Hadoop cluster and different big data analytic tools including Map Reduce, Hive, and pig.
• Involved in loading data from LINUX file system, servers, and Java web services.
• Involved in creating Hive tables, loading with data and writing hive queries which runs internally in Map Reduce way.
• Developed the Map Reduce programs to parse the raw data and store the pre-Aggregated data in the partitioned tables.
• Loaded and transformed large sets of structured, semi structured, and unstructured data with Map Reduce, Hive and pig.
• Developed Map Reduce programs in Java for parsing the raw data and populating staging Tables.
• Experienced in developing custom input formats and data types to parse and process unstructured and semi structured input data and mapped them into key value pairs to implement business logic in MapReduce.
• Involved in using HCATALOG to access Hive table metadata for MapReduce or Pig code.
• Experience in implementing custom sterilizer, interceptor, source and sink as per the requirement in flume to ingest data from multiple sources.
• Experience in setting up Fan-out workflow in flume to design v shaped architecture to take data from many sources and ingest into single sink.
• Developed Shell scripts to automate and provide Control flow to Pig scripts.
• Exporting of result set from HIVE to MySQL using Sqoop export tool for further processing.
• Evaluated usage of Oozie for Workflow Orchestration.
• Converted unstructured data to structured data.
• Automation of all the jobs starting from pulling the Data from different Data Sources like MySQL and pushing the result dataset to Hadoop Distributed File System and running MR, PIG, and Hive jobs using Oozie (Work Flow management).
• Worked on No-SQL databases like Cassandra for POC purpose in storing images and URIs.
• Integrating bulk data into Cassandra file system using MapReduce programs.
• Worked on MongoDB for distributed storage and processing.
• Designed and implemented Cassandra and associated Restful web service.
• Implemented Row Level Updates and Real time analytics using CQL on Cassandra Data.
• Used Cassandra CQL with Java API's to retrieve data from Cassandra tables.
• Worked on analyzing and examining customer behavioral data using Cassandra.
• Created partitioned tables in Hive.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Involved in cluster setup, monitoring, test benchmarks for results.
• Involved in agile methodologies, daily scrum meetings, spring planning's.

HIVE Developer

Confidential, McLean, VA

Responsibilities:

• Developed custom data Ingestion adapters to extract the log data and click stream data from external systems and load into HDFS.
• Creating Hive tables, loading data and writing hive queries for building Analytical Datasets.
• Worked on various components of Hadoop Ecosystem Map Reduce, Hive, Pig, Sqoop, Impala, Flume, Oozie, Hbase, and YARN.
• Hands-n Experience in working with Cloudera platform.
• Knowledge in installing, configuring and using Hadoop Ecosystem components.
• Proficient in Hive Query language and experienced in hive performance Optimization using Partitioning, Dynamic-Partitioning and bucketing concepts.
• Used Impala for data analysis.
• Hands-n experience in using the data ingestion tools Sqoop and Flume.
• Collected the log data from various sources (web servers, Application servers, and customer devices) using Flume and stored in HDFS to perform various analysis.
• Performed Data transfer between HDFS and their Relational Database Systems (MySQL, SQL Server, Oracle and DB2) using Sqoop.
• Knowledge in installation, configuration, supporting and managing Hadoop Clusters using Apache, Cloudera (CDH3, CDH4) distributions.
• Strong experience in database design, writing complex SQL Queries.
• Experience in working with an onsite-offshore model.
• Good Data Warehouse experience in MS SQL.
• Good experience in Linux, Mac, Windows environment.
• Used various development tools like Eclipse, and Android Studio.
• Knowledge of Cloudera Hadoop and Map-Reduce distribution components and their custom packages.
• Worked on real time data ingestion and processing using Hbase.
• Designed and developed Job flows using Oozie.
• Developed Sqoop commands to pull the data from different resources.
• Written Hive jobs to parse the logs and structure them in tabular format to facilitate effective querying on the log data.
• Experienced in OLAP analysis and ETL process.
• Applied transformations and loaded into HBase for data processing.
• Worked with Shell scripting.
• Hands-on experience in Hive Query Optimization.
• Knowledgeable in Hadoop Storage system.
• Knowledgeable in ETL tools like SAP Data server, SAS Data Management, and QlikView.
• Worked with data lakes.

Big Data Hadoop Developer

Confidential, Kansas City, MO

Responsibilities:

• Launching and Setup of Hadoop Cluster which includes configuring different components of Hadoop.
• Hands on experience in loading data form UNIX file system to HDFS.
• Cluster coordination services through Zookeeper.
• Installed and configured Flume, Hive, Pig, Sqoop and Oozie n the Hadoop cluster.
• Involved in creating Hive tables, loading data and running hive queries on these data.
• Extensive Working knowledge of partitioned table, UDFs, performance tuning, compression-related properties, thrift server in Hive.
• Building a framework for string and processing input data from various resources.
• Involved in writing optimized Pig Script along with involved in developing and testing Pig Latin Scripts.
• Developed multiple MapReduce jobs in java for data cleaning & preprocessing.
• Experienced in defining job flows.
• Maintaining job status and configuration in a relational table (MySQL) for tracking and string them in HBase.

Hadoop developer

Confidential

Responsibilities:

• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
• Developed Simple to complex Map reduce Jobs using Hive and Pig.
• Optimized Map Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
• Handled importing of data from various data sources, performed transformations using Hive, Map Reduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop.
• Exported the analyzed data to the relational databases using Sqoop for the BI team to start the visualization process.
• Extensively used Pig for data cleansing.
• Created partitioned tables in Hive.
• Involved in creating Hive tables, loading with data and writing hive queries.
• Used Hive to analyze the partitioned and bucketed dat.
• Installed and configured Pig and written Pig Latin scripts.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Load and transform large sets of structured, semi structured and unstructured data.
• Responsible to manage data coming from different sources.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.