SUMMARY:

• Over 15 years of extensive experience in the field of information Technology having in depth knowledge in Leading, EAI, SOA, B2B/B2C, Big Data, object oriented design, architect, development, maintenance and 24/7 production support.
• Cloudera certified Hadoop developer with 5 Years of experience in Installing, configuring and using Hadoop ecosystem components including MapReduce, HDFS, Hbase, ZooKeeper, Oozie, Hive, Cassandra, Sqoop, Pig and Flume.
• Expert in installation, configuration, deployment of packages and updates in and MapR Hadoop environment.
• Expert level of programming skills includes Java, C/C++ and SOA.
• Expert in database designing and programming using MS SQL server, T - SQL, Oracle, PL/SQL, DB2, Sybase, MySQL, NoSQL (HBase, MongoDB, Cassandra).
• Provided solution and NoSQL development using Cassandra, HBase, Spark and Hive.
• Good understanding of Classic Hadoop and Yarn architecture along with various Hadoop Demons such as Job Tracker, Task Tracker, Name Node, Data Node, Secondary Name Node, Resource Manager, Node Manager, Application Master and Containers.
• Expert lever of scripting using Pig scripts and Hive queries for processing and analyzing large volume of data.
• Expert in importing and exporting of data using Sqoop from HDFS to relational database system and vice-versa.
• Expert in web services like REST and SOAP along with XML and JSON data formats.
• Expert in ETL processing and writing queries for generating multidimensional data.
• Expert in various development tools such as Eclipse, Visual Studio and Net Beans.
• Expert in understanding of Data Structures and Algorithms for Optimization.
• Expert in Software Development Life Cycle (SDLC).
• Working experience on classic and yarn distributions of Hadoop like the Apache Hadoop 1.0.4, ClouderaCHD3 and CHD4.
• Provided Technical direction for development design and systems integration for client management from requirements analysis to implementation.
• Expert level program solving skills. Ability to prioritize workload and consistency meet deadlines.
• Self-motivated, work independently, self-learner, good listener, excellent communication, interpersonal and decision making skills.

PROFESSIONAL EXPERIENCE:

Confidential, San Ramon, CA

Data Engineer

Responsibilities:

• Provided solution to build prototype and POC using Docker container.
• Participated in gathering and analysis of business and technical requirements.
• Formulated and executed designing standards for data analytical systems.
• Provided solution for data transfer using Sqoop from RDBMS such as MySQL into Hive and MapR HDFS.
• Provided solutions for creating packages using scripts and MakeFile for deployment and installation.
• Analysis, visualization and algorithm implementation using R, PySpark and Python.
• Implemented data ingestion process using Talend.
• Implemented High Availability and replication using HVR, PostgreSQL, Greenplum and NFS.
• Provided solution for in memory cache using EhCache using Spring and Java.
• Provided solution for query tuning using Greenplum and PostgreSQL.
• Integrated the Hive warehouse with HBase for information sharing among teams.
• Designed and created Hive external tables using shared Meta store and supported partitioning, dynamic partitioning for faster data retrieval.
• Deployed and supported MapReduce programs that are running on the cluster.
• Testing using HiveQL scripts for automation of create, load tables for extraction in an MapR distribution.
• Involved in search engine development using Cassandra, Lucene, SOLR and AWS Cloud.
• Provided solution using Scala collections Array, Map, Tuple, List and ArrayBuffer.
• Provided solution using REPL and creating RDD and DataFrames using Scala.
• Provided solution for Spark context, Spark Streaming and MLlib using Scala.
• Provided solution for GraphX, Spark SQL and Performance tuning using Scala.
• Provided solution in building spark projects using Scala and SBT.
• Involved in installation, configuration and securities for access in a MapR Hadoop ecosystem.
• Provided solution in extraction of data from Oracle to Hbase using Sqoop job.
• Provided solution in defining job flow and workflow using Oozie.
• Provided solution using MongoDB and Cassandra databases.
• Provided solution using Hadoop ecosystem HDFS, MapReduce, Pig, Hive, Impala, HBase, and Zookeeper.
• Provided solution to Install, configure, adding and removing noes to the cluster.
• Provided solution using monitoring tools such as Ganglia, Nagios and Check MK.
• Development of robust, high volume and scalable applications.
• Implementation of best practices in Architecture, design and clean code.
• Implementation of real time data collection and processing.
• Involved in design and ensure for modular, portable and performance optimized.
• Worked with limited supervision in an Agile/DevOps/CICD environment.

Environment: Hadoop, Hortonworks 2.3, MapR, Kafka 1.2.0, Storm, Eclipse, Oozie, Impala, Zookeeper, JSON, Agile/DevOps, MongoDB, Java, Tableau, Maven, Jira, Confluence, Kerberos.

Confidential, Fort Worth, TX

Data Engineer/Hadoop Architect

Responsibilities:

• Provided technical guidance with SDLC design for enterprise applications.
• Participated in gathering and analysis of business and technical requirements.
• Formulated and executed designing standards for data analytical systems.
• Involved in setting up of the 64 node cluster and configuring the Hadoop platform.
• Migrating the data form MySQL into HDFS using Sqoop and importing various formats of flat files into HDFS.
• Testing using HiveQL scripts for automation of create, load tables for extraction.
• Involved in search engine development using Cassandra, Lucene, SOLR and AWS Cloud.
• Involved in installation, configuration and deployment of packages in an MapR Hadoop distribution.
• Provided solution in extraction of data from Oracle to Hbase using Sqoop job.

Environment: Hadoop, Hortonworks 2.3, MapR, Kafka 1.2.0, Storm, Eclipse, Oozie, Impala, Zookeeper, JSON, NFS, Agile/DevOps, MongoDB, Java, Tableau, Maven, Jira, Confluence, Kerberos.

Confidential, Dallas, TX

Data Engineer

Responsibilities:

• Installed and configured Hadoop, MapReduce, HDFS, developed multiple MapReduce jobs using Java for data cleaning and processing.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Cluster coordination services using ZooKeeper.
• Provided solution using Hadoop ecosystem
• HiveQL scripts to analyze member data to determine member demographics.
• Implementation of messaging using Storm and Kafka for building data lake.
• Provided solution for MapReduce development using Hadoop.
• Provided solution for hardware and architectural guidance, planning, estimating cluster capacity and creating roadmaps for Hadoop cluster deployment.

Environment: Hadoop, MapReduce, HDFS, Hive, Spark, Python, Storm, Java (JDK 1.6), Spring, Pig, Zookeeper, Kerberos, Agile.

Confidential, Milwaukee, WI

Data Engineer

Responsibilities:

• Provided solution using Hive and Pig.
• Provided solution using Hadoop ecosystem
• Created Hive tables to store the processed results in tabular format.
• Create HBase tables to store variable data formats of data coming from different applications.
• Managing and scheduling jobs on Hadoop cluster using Oozie.
• End to end performance tuning of Hadoop clusters and MapReduce routines against very large datasets.
• Provided solution for designing automated state machine using Python/C++.
• Good understanding of ETL tools and how they can be applied in Big Data environment.
• Provided solution using open source technologies.
• De batching and processing the HL7 EDI file.

Environment: MapReduce, Hive, Pig, Sqoop, Oozie, Oracle, Kerberos, Cloudera, Java, Python, Agile, HL7.