SUMMARY

• Over 8 years of programming and software development experience with skills in data analysis, de - sign, development, testing and deployment of software systems from development stage to production stage in Big Data and Java technologies.
• 4+ years of experience in Big Data and tools in Hadoop Ecosystem including Pig, Hive, Sqoop, Oozie, Zookeeper, HBase, Spark, Flume, Storm and Kafka.
• Good experience in Cloudera Distribution (CDH), Horton works Distribution.
• Excellent knowledge on Hadoop Ecosystem Architecture and components such as Hadoop Distributes File System (HDFS), MRv1, MRv2 (YARN), Job Tracker, Task Tracker, Name Node, Data Node, Resource Manager, Node Manager and Map Reduce programming.
• Strong knowledge of Software Development Life Cycle (SDLC) and expertise in detailed design documentation.
• Excellent analytical, logical, debugging skills, programming skills, Self-Motivated, Quick Learner, & Team Player
• Experience in analyzing data using Hive (Hive QL), Pig Latin, HBase & Map Reduce programs (Java).
• Experience in extending Hive and Pig core functionality by writing User defined Functions (UDF).
• Experience in migrating the data using Sqoop from HDFS to Relational Database System and vice-versa.
• Good understanding on NoSQL databases and hands on work experience in writing applications on NoSQL databases like HBase, Cassandra.
• Extensive experience in UNIX Shell Scripting and have good knowledge of Python.
• Knowledge of installing, configuring, debugging and troubleshooting Hadoop clusters.
• Good Exposure on Apache Hadoop Map Reduce programming, PIG Scripting and Distribute Application and HDFS.
• Used Oozie workflow engine to manage interdependent Hadoop jobs and to automate several types of Hadoop jobs such as Java Map Reduce, Hive and Sqoop as well as system specific jobs.
• Experience in managing Hadoop clusters using Cloudera Manager Tool.
• Experience in implementing different transformations on hive data by implementing Spark SQL.
• Experience in implementing in setting up standards and processes for Hadoop based application design and implementation.
• Worked extensively with Apache Hive, partitioning, bucketing and writing UDFS, UDAFS, and UDTFS in Java for Hive.
• Experience in fixing bugs, tracking defects, issues & risks using Quality Center.
• Having Experience on Development tools like Eclipse.
• Experience on Source control repositories like SVN.
• Scripting to deploy monitors, checks and critical system admin functions automation.
• Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
• Familiar with importing and exporting data using Sqoop from RDBMS MySQL, Oracle, Teradata and also using fast loaders and connectors experience.
• Sqoop jobs, PIG and Hive scripts were created for data ingestion from relational databases to compare with historical data.
• Developed Spark scripts by using Scala Shell commands as per the requirement.
• Load the data into Spark RDD and do in memory data Computation to generate the Output response.
• Created data ingestion plans for loading the data from external sources using Sqoop.
• WroteStormtopology to accept the events from Kafka producer and emit into Cassandra DB.
• Developed Kafka producer and consumers, Spark andHadoopMap Reduce jobs.
• Familiar with data architecture including data ingestion pipeline design,Hadoopinformation architecture, data modeling and data mining, machine learning and advanced data processing.

TECHNICAL SKILLS

Big Data Technologies: HDFS, Map Reduce, Hive, Pig, Sqoop, Flume, Oozie, Avro, Hadoop Streaming, Zookeeper, Impala, and Apache Spark.

Hadoop Distributions: Cloudera (CDH4/CDH5),Horton Works

Languages: Java, C, SQL,PL/SQL,PIG-Latin, HQL

IDE Tools: Eclipse, Net Beans, RAD

Framework: Hibernate, Spring, Struts, Junit

Web Technologies: HTML5, CSS3, JavaScript, JQuery, AJAX, Servlets, JSP,JSON, XML, XHTML

Operating Systems: Windows (XP,7,8,10), UNIX, LINUX, Ubuntu, CentOS

Application Servers: JBoss, Tomcat, Web Logic

Reporting Tools /ETL Tools: Tableau, Power view for Microsoft Excel

Databases: Oracle, MySQL, DB2, Derby, PostgreSQL, No-SQL Database (Hbase, Cassandra)

PROFESSIONAL EXPERIENCE

Hadoop Developer

Confidential, Ashburn, VA

Responsibilities:

• Installed and configured Hadoop Map reduce, HDFS, Developed multiple Map Reduce jobs in java for data cleaning and preprocessing.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Experience in defining job flow using Oozie and shell scripts.
• Experienced in implementing various customizations in Map Reduce at various levels by implementing custom input formats, custom record readers, partioners, and data types in java.
• Experience in ingesting data using flume from web server logs and telnet sources.
• Implemented encryption and masking on customer sensitive data in flume by building a custom interceptor and masking and encrypting the data as per the requirement by considering the rules in MySQL.
• Experience in managing and reviewing Hadoop log files.
• Extracted files from RDBMS through Sqoop and placed in HDFS and processed.
• Experience in running Hadoop streaming jobs to process terabytes of xml format data.
• Supported Map Reduce Programs those are running on the cluster.
• Involved in loading data from UNIX file system to HDFS.
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in Map Reduce way.
• Experienced in building the Kafka producers and consumers in order to pull data and ingest the data into Hbase.
• Experience in integrating Kafka and spark for data processing on the fly by developing Kafka spouts and taking the data into storm bolts and implement some processing in bolt and insert into Hdfs or Hbase depending on the requirement.
• Experience in implementing Hive-Hbase integration by creating hives external tables and using Hbase storage handler.
• Designed and implemented Map Reduce based large scale parallel relation learning system.
• Developed Master tables in HIVE using a “json deserializer” or “get json object” or “json tuple” functions of HIVE.
• Designed the entire flow in HDFS in such a way that is needed to be achieved using Oozie workflows.
• Designed and developed RDD Seeds using Scala and cascading.
• Generating Scala and java classes from the respective APIs so that they can be incorporated in the overall application
• Writing Scala classes to interact with the database.
• Writing user console page in lift along with the snippets in Scala.

Environment: Java, Eclipse, Oracle 10g, Sub Version, Hadoop, Hive, HBase, Map Reduce, HDFS, Pig Hive, Java (JDK 1.6), Scala Hadoop Distribution of Cloudera, Map Reduce, UNIX Shell Scripting.

Hadoop Developer

Confidential, Wichita, KS.

Responsibilities:

• Worked on writing transformer/mapping Map-Reduce pipelines using Java.
• Involved in creatingHive Tables, loading with data and writing Hive queries which will invoke and run Map Reduce jobs in the backend.
• Involved in loading data into HBase using HBase Shell, HBase Client API, Pig and Sqoop.
• Designed and implemented Incremental Imports into Hive tables.
• Worked in Loading and transforming large sets of structured, semi structured and unstructured data
• Involvedin collecting, aggregating and moving data from servers to HDFS using Apache Flume
• Written Hive jobs to parse the logs and structure them in tabular format to facilitate effective querying on the log data
• Involved in creating Hive tables, loading with data and writing hive queries that will run internally in map reduce way.
• Experienced in managing and reviewing theHadooplog files.
• Migrated ETL jobs to Pig scripts do Transformations, even joins and some pre-aggregations before storing the data onto HDFS.
• Implemented the workflows using Apache Oozie framework to automate tasks
• Worked on different file formats like Sequence files, XML files and Map files using Map Reduce Programs.
• Involved in Unit testing and delivered Unit test plans and results documents using Junit and MRUnit.
• Developed scripts and automated data management from end to end and sync up between all the clusters.
• Involved in Setup and benchmark of Hadoop /HBase clusters for internal use.
• Setup Hadoop cluster on Amazon EC2 using whir for POC
• Created and maintained Technical documentation for launching HADOOP Clusters and for executing pig Scripts.
• Developed Spark scripts by using Scala Shell commands as per the requirement.
• Load the data into Spark RDD and do in memory data Computation to generate the Output response.
• Developed Spark code using Scala and Spark-SQL/Streaming for faster testing and processing of data.
• Exploring with the Spark improving the performance and optimization of the existing algorithms in Hadoopusing Spark Context, Spark-SQL, Data Frame, Pair RDD's, Spark YARN.

Environment: Hadoop, Big Data, HDFS, Map Reduce, Sqoop, Oozie, Pig, Hive, Hbase, Flume, LINUX, Java, Eclipse, Cassandra, Hadoop Distribution of Cloudera, PL/SQL, SQL*PLUS, Toad 9.6, Windows NT, UNIX Shell Scripting, Spark, Putty and Eclipse

Hadoop Developer

Confidential

Responsibilities:

• Analyzed large data sets by running Hive queries and Pig scripts.
• Worked with the Data Science team to gather requirements for various data mining projects.
• Involved in creating Hive tables, and loading and analyzing data using hive queries.
• Developed Simple to complex Map Reduce Jobs using Hive and Pig.
• Worked with application teams to install operating system,Hadoopupdates, patches, version upgrades as required.
• Developed multiple Map Reduce jobs in java for data cleaning and preprocessing.
• Involved in loading data from LINUX file system to HDFS.
• Responsible for managing data from multiple sources Extracted files from MySQL through Sqoop and placed in HDFS and processed.
• Involved in HDFS maintenance and loading of structured and unstructured data.
• Installed and configured Pig and also written Pig Latin scripts.
• Written Hive queries for data analysis to meet the business requirements.
• Involved in HDFS maintenance and loading of structured and unstructured data.
• Imported data using Sqoop to load data from MySQL to HDFS on regular basis.
• Developing Scripts and Batch Job to schedule variousHadoopProgram.
• Written Hive queries for data analysis to meet the business requirements.

Environment: Hadoop, HDFS, Pig, Hive, Map Reduce, Sqoop, LINUX, and Big Data

Java Developer

Confidential

Responsibilities:

• Involved in Analysis and Design of the Project, which is based on MVC (Model-View-Controller) Architecture and Design Patterns.
• Involved in developing prototypes of the product.
• Developed activity, sequence and class diagrams using Unified Modeling Language and Rational Rose.
• Designed GUI Components using Tiles frame work and Validation frame work.
• Developed Controller Servlets, Action and Form objects for process of interacting with Oracle database and retrieving dynamic data.
• Responsible for coding SQL Statements and Stored procedures for back end communication using JDBC.
• The application was designed on Struts Framework to make use of MVC pattern.
• Used Apache Log4J logging API to log errors and messages.
• Developed XML parser for File parsing.
• Involved in writing Detail Design Documents with UML Specifications.
• Involved in unit testing and system testing and also responsible for preparing test scripts for the system testing.
• Developed backend components, DB Scripts for the backend communication.
• Used Cognos to create reports, which were mailed to be mailed to manager every hour.
• Responsible for performance tuning of the product and eliminating memory leakages in the product.
• Conducting Code Review meetings with developers.

Environment: Java, JSP, Servlets, JDBC, LOG4J, Oracle, XML, HTML, Cognos.

Java Developer

Confidential

Responsibilities:

• Worked on Requirement analysis, gathered all possible business requirements from end users and business Analysts
• Involved in creation of UML diagrams like Class, Activity, and Sequence Diagrams using modeling tools of IBM Rational Rose
• Worked with core Java code extensively using interfaces and multi-threading techniques
• Involved in production support and documenting the application to provide training and knowledge transfer to the user
• Used Log4j for logging mechanism and developed wrapper classes to configure the logs
• Used JUnit and Test cases for testing the application modules
• Developed and configured the Java beans using Spring MVC framework
• Developed the application using Rational Team Concert and worked under Agile Environment
• Developed SQL stored procedures and prepared statements for updating and accessing data from database
• Also used C++ to create some libraries used in the application

Environment: C++, Java, JDBC, Servlets, JSP, Struts, Eclipse, Oracle 9i, Apache Tomcat, CVS, JavaScript, Log4J