SUMMARY

• 8+ years of professional experience in IT in Analysis, Design, Development, Testing, Documentation, Deployment, Integration, and Maintenance of web based and Client/Server applications using SQL and Big Data technologies.
• 8+ years of experience in Application Development using Hadoop and related Big Data technologies such as HBASE, HIVE, PIG, FLUME, OOZIE, SQOOP, and ZOOKEEPER.
• In - depth Knowledge of Data Structures, Design and Analysis of Algorithms and having good understanding of Data Mining and Machine Learning techniques.
• Excellent knowledge on Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node.
• Well versed in installation, configuration, supporting and managing of Big Data and underlying infrastructure of Hadoop Cluster.
• Having Hadoop/Big Data related technology experience in Storage, Querying, Processing and analysis of data.
• Proficient in design and development of Map Reduce Programs using Apache Hadoop for analyzing the big data as per the requirement.
• Hands on experience in installing, configuring, and using Hadoop ecosystem components like HDFS, MapReduce, HBase, Zookeeper, Oozie, Hive, Sqoop, Pig, and Flume.
• Skilled in writing Map Reduce jobs in Pig and Hive.
• Knowledge in managing and reviewing Hadoop Log files.
• Expertise in wide array of tools in the Big Data Stack such as Hadoop, Pig, Hive, HDFS, MapReduce, Sqoop, Spark, Kafka, Yarn, Oozie, and Zookeeper.
• Knowledge in Streaming the Data to HDFS using Flume.
• Excellent programming skills with experience in Java, C, SQL and Python Programming.
• In depth and extensive knowledge of analyzing data using HiveQL, Pig Latin, HBase and custom Map Reduce programs in Java.
• Experience in tuning and troubleshooting performance issues in Hadoop cluster.
• Worked on importing data into HBaseusing HBase Shell and HBase Client API.
• Hands on experience in using Sqoop to import data into HDFS from RDBMS and vice-versa.
• Extensive experience working on various databases and database script development using SQL and PL/SQL
• Hands on experience in application development using Java, RDBMS and Linux Shell Scripting.
• Experience in writing Pig and Hive scripts and extending Hive and Pig core functionality by writing custom UDFs.
• Knowledge in writing live Real-time Processing using Spark Streaming with Kafka.
• Involved in HBase setup and storing data into HBase, which will be used for further analysis.
• Good experience in Hive partitioning, bucketing and perform different types of joins on Hive tables and implementing Hive Sere like JSON and Avro.
• Supported Map Reduce Programs running on the cluster and wrote custom Map Reduce Scripts for Data Processing in Java.

TECHNICAL SKILLS

Hadoop Ecosystem: HDFS, MapReduce, HIVE, PIG, Sqoop, Flume, Oozie, Kafka, Zookeeper, Avro, Spark, Storm.

Programming Languages: C, C#, Java, SQL, Scala, PL/SQL, Python, Linux shell scripts

Web Technologies: HTML, XML, JDBC, JSP, JavaScript, AJAX, SOAP

Web Servers: Web Logic, Web Sphere, Apache Tomcat

Tools: Used Eclipse, IntelliJ, GIT, Putty, WinSCP

Database: Oracle 11g/10g,DB2, Teradata, Vertica, MySQL.

NoSQL Databases: MongoDB, Cassandra, HBase, DynamoDB.

Testing: Hadoop MRUNIT Testing, Quality Center, Hive Testing

Frameworks: MVC, Struts 2/1, Hibernate 3, Spring 3/2.5/2

Monitoring and Reporting tools: Ganglia, Nagios, Custom Shell scripts, Tableau.

ETL Tools: Informatica, Pentaho.

Operating System: Ubuntu (Linux), Windows, RedHat

PROFESSIONAL EXPERIENCE:

Confidential, MN

Hadoop Developer

Responsibilities:

• Worked with Business Analyst and helped representing the business domain details.
• Hands on experience in gathering information from different nodes into Greenplum database and then Sqoop incremental load into HDFS.
• Experience in importing the real-time data to Hadoop using Kafka and implemented the Oozie job.
• Involved in loading data from LINUX file system to HDFS
• Experience in Writing Map Reduce jobs for text mining and worked with predictive analysis team to check the output and requirement.
• Hands on experience in writing hive UDF's for the requirements and to handle different schema’s and xml data.
• Used Pig as ETL tool to do transformations, event joins, filter both traffic and some pre-aggregations before storing the data onto HDFS.
• Wrote Hive and Pig scripts for joining the raw data with the lookup data and for some aggregative operations as per the business requirement.
• Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
• Involved in writing Flume and Hive scripts to extract, transform and load the data into Database
• Implemented Partitioning and bucketing in Hive based on the requirement.
• Connected Tableau from client end with AWS ip addresses and view the end results.
• Coordinator and Oozie workflows are developed to automate Hive, Map Reduce, Pig and other jobs.
• Creation of test cases as part of enhancement rollouts and Involved in Unit level and Integration level testing.
• Hands on experience in working with snappy compression and also different file formats.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports For the BI team.

Environment: Hadoop, Map Reducer, Cloudera Manager, HDFS, Hive, Pig, Sqoop, Spark, Oozie, Impala, Greenplum, Kafka, SQL, Java (jdk 1.6), Eclipse.

Confidential, Sunnyvale, CA

Hadoop Developer

Responsibilities:

• Worked on analyzing, writing HadoopMapReduce jobs using JavaAPI, Pig and Hive.
• Responsible for building scalable distributed data solutions using Hadoop.
• Involved in loading data from edge node to HDFS using shell scripting.
• Created HBase tables to store variable data formats of PII data coming from different portfolios.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
• Worked with using different kind of compression techniques to save data and optimize data transfer over network using LZO, Snappy, Bzip etc.
• Analyze large and critical datasets using Cloudera, HDFS, HBase, MapReduce, Hive, HiveUDF, Pig, Sqoop, Zookeeper, &Spark.
• Developed custom aggregate functions using SparkSQL and performed interactive querying.
• Used Scoop to store the data into HBase and Hive.
• Worked on installing cluster, commissioning & decommissioning of DataNode, NameNode high availability, capacity planning, and slots configuration.
• Creating Hive tables, dynamic partitions, buckets for sampling, and working on them using HiveQL.
• Used Pig to parse the data and Store in Avro format.
• Stored the data in tabular formats using Hive tables and Hive Serdes.
• Collecting and aggregating large amounts of log data using ApacheFlume and staging data in HDFS for further analysis.
• Worked with NoSQL databases like HBase for creating HBase tables to load large sets of semi structured data coming from various sources.
• Implemented a script to transmit information from Oracle to HBase using Sqoop.
• Implemented MapReduce programs to handle semi/unstructured data like XML, JSON, and sequence files for log files.
• Fine-tunedPigqueries for better performance.
• Involved in writing the shell scripts for exporting log files to Hadoop cluster through automated process.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
• Installed Oozie workflow engine to run multiple Hive and pig jobs.
• Analyzed large amounts of data sets to determine optimal way to aggregate and reporton it.

Environment: Hadoop, MapReduce, HDFS, Yarn, Sqoop, Oozie, Pig, Hive, HBase, Spark, Java, Eclipse, UNIX shell scripting, python, Hortonworks.

Confidential - Bethlehem, PA

UI Developer

Responsibilities:

• Worked closely with the business analysts to convert the Business Requirements into Technical Requirements and prepared low and high-level documentation.
• Hands on experience on writing MR jobs for encryption and also for converting text data into Avro format.
• Hands on experience in joining raw data with the data using Pig scripting.
• Hands on experience in writing scripting for copying data between different clusters and also between different Unix file systems.
• Hands on experience in writing MR jobs for cleansing the data and to copy it to AWS cluster form our cluster.
• Developed Spark SQL script for handling different data sets and verified its performance over MR jobs.
• Connected Tableau from client end with AWS ip addresses and view the end results.
• Developed Coordinator and Oozie workflows to automate the jobs.
• Hands on experience in writing hive UDF's to handle different Avro schema's.
• Experience with moving large datasets hourly with AVRO file format and imposing hive and impala queries.
• Hands on experience in working with snappy compression and also different file formats.
• Developed shell script to back up the name node meta data.
• Cloudera Manger was used to Monitor the health of Jobs which are running on the cluster.

Environment: Hadoop, Map Reducer, Cloudera Manager, HDFS, Hive, Pig, Sqoop, Spark, Oozie, Impala, Greenplum, Kafka, SQL, Java (jdk 1.6), Eclipse.

Confidential

JAVA Developer

Responsibilities:

• Worked in Java, JDBC, My SQL, Jsp, Java Script, AJAX.
• Involved in design (UML), development, coding, validation & verification of Purchase Order subsystem.
• Involved in developing the Delivery Challan and Sales Invoice module.
• Involved in designing Admin front page
• Designing Login screen of front page.
• Developing Junits and testing.

Environment: Java, Servlets, Jdbc, JSP, Java Script, My SQL, Tomcat, Windows NT.

Confidential

JAVADeveloper

Responsibilities:

• Involved in various stages of Enhancements in the Application by doing the required analysis, development, and testing.
• Prepared the High and Low-level design document and Generating Digital Signature
• For analysis and design of application created Use Cases, Class and Sequence Diagrams.
• For the registration and validation of the enrolling customer developed logic and code.
• Developed web-based user interfaces using struts frame work.
• Handled Client-side Validations used JavaScript and
• Involved in integration of various Struts actions in the framework.
• Used Validation Framework for Server-side Validations
• Created test cases for the Unit and Integration testing.
• Front-end was integrated with Oracle database usingJDBCAPI through JDBC-ODBC Bridge driver at server side.

Environment: Java Servlets, JSP, Java Script, XML, HTML, UML, Apache Tomcat, JDBC, Oracle, SQL.