SUMMARY:

• Over 8 years of combined experience in IT industry, played major role in implementing, developing and maintenance of various Web Based applications using Java, J2EE Technologies and Big Data Ecosystem.
• Excellent understanding of Hadoop architecture and complete understanding of Hadoop daemons and various components such as HDFS, YARN, Resource Manager, Node Manager, Name Node, Data Node and Map Reduce programming paradigm.
• Strong hands on experience in Hadoop Framework and its ecosystem including but not limited to HDFS Architecture, MapReduce Programming, Hive, Pig, Sqoop, HBase, Oozie,Hue with CDH3&4and 5Xclusters
• Good knowledge in writing Spark application using Python, Scala and Java.
• Good experience transformation and storage: HDFS, MapReduce, Spark
• Experience in troubleshooting errors in HBase Shell/API, Pig, Hive and MapReduce.
• Extensive hands on experience in writing complex MapReduce jobs, Pig Scripts and Hive data modeling
• Experience in importing and exporting data between HDFS and Relational Database Management systems using Sqoop.
• Collected logs data from various sources and integrated in to HDFS using Flume.
• Extensively implemented POC's on migrating to Spark - Streaming to process the live data
• Worked on parsing Jason data.
• Worked on Data serialization formats for converting complex objects into sequence bits by using Avro, Parquet, JSON, CSV formats.
• Architected, Designed and maintained high performing ELT/ETL Processes.
• Experience in loading data to HDFS from UNIX (Ubuntu, Fedora, Centos) file system.
• Hands on experience with J-Unit and Log4j in developing test cases and determining application functionality
• Accomplished with creating Test Plans, defining Test Cases, reviewing and maintaining Test Scripts, interacting with team members in fixing errors and executing Integration testing (SIT), User Acceptance Testing (UAT), Stage (PFIX) Unit, System Integrated Test, Regression Test and Customer Test.
• Very good Experience in creating build scripts using Maven and Ant.
• Strong command over relational databases: MySQL, Oracle, SQL Server and MS Access.
• Experience in Web Services using XML, HTML.
• Good experience in writing SQL queries, stored procedures, functions, triggers and packages.
• Experience in developing Client Side Web applications using HTML, JSP, JQuery, JSTL, AJAX, andCustom Tags while implementing the client side validations using JavaScript and Server side validations using Struts Validations Framework.
• Having good knowledge of Oracle9i, 10g, 11g as Database and excellent in writing the SQL queries and scripts.
• Worked on setting up and configuring AWS's EMR Clusters and Used Amazon IAM to grant fine-grained access to AWS resources to users
• Experience in Building S3 buckets and managed policies for S3 buckets and used S3 bucket and Glacier for storage and backup on AWS
• Experience in preparing Design Documentation.
• Good Team Player, quick learner and self-starter with effective communication, motivation and organizational skills combined with attention to detail and business process improvements
• Ability to handle a team of developers and co-ordinate smooth delivery of the project.
• Problem solving capability peered with strong communication skills.
• Ability to learn and implement new technologies faster.

TECHNICAL SKILLS:

Big Data Ecosystem: Hadoop, MapReduce, Pig, Hive, YARN, Kafka, Flume, Sqoop, Impala, Oozie, ZooKeeper, Spark, Ambary, Mahout, MongoDB, Cassandra, Avro, Parquet and Snappy.

Hadoop Distributions: Cloudera (CDH3, CDH4, and CDH5), Hortonworks, AWS

Languages:  Java, Python, Ruby, SQL, HTML, DHTML, Scala, JavaScript, XML and C/C++

No SQL Databases: Cassandra and HBase

Java Technologies: Servlets, JSP, JDBC, JNDI, Java Beans

XML Technologies: XML (SAX, DOM), JAXB

Web Design Tools: HTML, DHTML, AJAX, JavaScript, JQuery and CSS, Angular Js and JSON

Development / Build Tools: Eclipse, Ant, Maven, IntelliJ, JUNIT and log4J.

Frameworks: Struts, spring and Hibernate

App/Web servers: WebSphere, WebLogic, JBoss and Tomcat

DB Languages: MySQL, PL/SQL, PostgreSQL and Oracle

RDBMS: Teradata, Oracle 9i,10g,11i, MS SQL Server, MySQL and DB2

Operating systems: UNIX, LINUX, Mac OS and Windows Variants

ETL Tools: Talend, Informatica, Pentaho

Testing: HP QC, JIRA, QTP, Load Runner

PROFESSIONAL EXPERIENCE:

Confidential, Dearborn, MI

Hadoop Developer/Admin

Responsibilities:

• Used Agile methodology in developing the application, which included iterative application development, weekly Sprints, stand up meetings and customer reporting backlogs.
• Worked on a live 60 nodes Hadoop cluster running CDH
• Worked with highly unstructured and semi structured data of 90 TB in size (TB of data with replication factor of 3)
• Worked on Hue interface for querying the data.
• Integrate Apache Spark with Hadoop components.
• Implemented Apache Spark data processing project to handle data from RDBMS and streaming sources.
• Designed batch processing jobs using Apache Spark to increase speeds by ten-fold compared to that of MR jobs.
• Developing design documents considering all possible approaches and identifying best of them.
• Responsible for building scalable distributed data solutions using Hadoop.
• Predicted consumer behavior, such as what products a particular user has bought and made predictions/recommendations based on recognizing patterns by using Hadoop, Hive and Pig queries.
• Installed and configured Hadoop, MapReduce, and HDFS.
• Developed multiple MapReduce jobs using Java API for data cleaning and preprocessing.
• Importing and exporting data into HDFS and HIVE from Oracle 11g database using Sqoop
• Responsible to manage data coming from different sources
• Monitoring the running MapReduce programs on the cluster.
• Responsible for loading data from UNIX file systems into HDFS.
• Installed and configured Hive.
• Installing, configuring and administering Jenkins CI tool using Chef on AWS EC2 instances
• Setting up servers through using AWS for deployment.
• Worked with application teams to install Hadoop updates, patches, version upgrades as required.
• Installed and configured Hive, Pig, Sqoop and Oozie on the CDH cluster.
• Involved in implementing High Availability and automatic failover infrastructure to overcome single point of failure for Name node utilizing zookeeper services.
• Implemented HDFS snapshot feature.
• Experience in Migrating Business reports to Spark, Hive, Pig and Map Reduce.
• Worked with big data developers, designers and scientists in troubleshooting map reduce job failures and issues with Hive, Pig and Flume.
• Involved in Installation and configurations of patches and version upgrades.
• Involved in Hadoop Cluster environment administration that includes adding and removing cluster nodes, cluster capacity planning, performance tuning, cluster Monitoring.
• Troubleshooting.
• Supported Map Reduce Programs those are running on the cluster.
• Involved in HDFS maintenance and administering it through Hadoop-Java API
• Involved in creating Hive Tables, loading with data and writing Hive queries which will invoke and run MapReduce jobs in the backend.
• Installed and configured Pig.
• Wrote Pig scripts to process unstructured data and create structure data for use with Hive.
• Developed the Sqoop scripts in order to make the interaction between Pig and MySQL Database.
• Developed scripts and automated data management from end to end and sync up b/w all the clusters.

Environment: Apache Hadoop 2.0.0, Pig 0.11, Hive 0.10, Sqoop 1.4.3, Flume, MapReduce, HDFS, LINUX, Oozie, Cassandra, Hue, H Catalog, Java, Eclipse, VSS, Red Hat Linux.

Confidential, Chicago, IL

Hadoop Engineer

Responsibilities:

• Responsible for installing, configuring, supporting and managing of Hadoop Clusters.
• Managed and reviewed Hadoop Log files as a part of administration for troubleshooting purposes.
• Monitoring and support through Nagios and Ganglia.
• Hands on experience in Python and Scala.
• Responsible for troubleshooting issues in the execution of MapReduce jobs by inspecting and reviewing log files.
• Used Scala to write MapReduce programs.
• Monitored workload, job performance and capacity planning using Cloudera Manager.
• Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop .
• Experience in setup, configuration and management of security for Hadoop clusters using Kerberos and integration with LDAP/AD at an Enterprise level.
• Used Hive and created Hive tables, loaded data from Local file system to HDFS.
• Production experience in large environments using configuration management tools like Chef and Puppet supporting Chef Environment with 500+ servers and involved in developing manifests. Developed Chef Cookbooks to manage systems configuration.
• Created user accounts and given users the access to the Hadoop cluster.
• Performed HDFS cluster support and maintenance tasks like adding and removing nodes without any effect to running nodes and data.
• Experience in configuring Java components using Spark.
• Responsible for HBase REST server administration, backup and recovery.
• Monitoring and controlling local file system disk space usage, log files, cleaning log files with automated scripts.
• Involved in writing OOZIE jobs for workflow automation.
• As a Hadoop admin, monitoring cluster health status on daily basis, tuning system performance related configuration parameters, backing up configuration xml files.
• Monitored all MapReduce Read Jobs running on the cluster using Ambary and ensured that they were able to read the data to HDFS without any issues.
• Involved in upgrading Hadoop Cluster from HDP 1.3 to HDP 2.0.
• Experience in configuring Java components using Spark.
• Involved in moving all log files generated from various sources to HDFS for further processing.
• Extracted the data from Teradata into HDFS using the Sqoop.
• Involved in collecting metrics for Hadoop clusters using Ganglia and Ambary.
• Prepared Oozie workflow engine to run multiple Hive and Pig jobs which run independently with time and data availability.
• Experience in developing the Pig UDFs to pre-process the data for analysis.
• Supported Data Analysts in running MapReduce Programs.
• Developed Hive queries to process the data and generate the data cubes for visualizing.
• Responsible for deploying patches and remediating vulnerabilities.
• Experience in setting up Test, QA, and Prod environment.
• Written Pig Latin Scripts to analyze and process the data.
• Involved in loading data from UNIX file system to HDFS.
• Created root cause analysis (RCA) efforts for the high severity incidents.
• Involved in Analyzing system failures, identifying root causes, and recommended course of actions.
• Used Kafka to publish messages.
• Worked hands on with ETL process. Handled importing data from various data sources, performed transformations.
• Coordinating with On-call Support if human intervention is required for problem solving
• Documenting the procedures performed for the project development.

Environment: Windows 2000/2003 Unix Linux Java, HDFS Map Reduce, Pig Hive HBase Flume Sqoop, NOSQL Oracle 9i/10g/11g RAC with Solaris/red hat, Shell Scripting, Apache Hadoop, Toad, MYSQL plus, HDFS, Hive, Sqoop, Zookeeper and HBase.

Confidential, Boston, MA

Hadoop Developer/Admin

Responsibilities:

• Involved in the Complete Software development life cycle (SDLC) to develop the application.
• Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, HBase database and Sqoop.
• Involved in loading data from LINUX file system to HDFS.
• Experience in managing and reviewing Hadoop log files.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Implemented test scripts to support test driven development and continuous integration.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.
• Installed and configured Hadoop MapReduce, HDFS, Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
• Created Pig Latin scripts to sort, group, join and filter the enterprise wise data.
• Involved in creating Hive tables, loading with data and writing Hive queries that will run internally in MapReduce way.
• Supported MapReduce Programs those are running on the cluster.
• Analyzed large data sets by running Hive queries and Pig scripts.
• Worked on tuning the performance Pig queries.
• Mentored analyst and test team for writing Hive Queries.
• Installed Oozie workflow engine to run multiple MapReduce jobs.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.

Environment: Hadoop 1.2.1, HIVE 0.11, PIG, Flume, Sqoop, Java, Linux, CentOS, RHEL 6, Ubuntu 13.04, Cloudera CDH4 and Puppet, Zookeeper and HBase.

Confidential, Charlotte,NC

Java Developer

Responsibilities:

• Worked with Agile Development Methodology with a Test-Driven approach.
• In case of application development to be loaded, modular approach was done.
• Depending on the business requirements, Class and Sequence diagrams were developed.
• Developed UML diagrams for analyzing the business requirements and designing the system.
• Was a part of every phase in the SDLC.
• Worked on developing JSP’s, Action Classes and delegating the tasks to the corresponding session parameters.
• Involved in Requirements Gathering, Analysis, Design and Development of the project.
• Developed services using Servlets, JSP’s, and Java Beans.
• Hibernate was the ORM tool used to relate the objects to the database.
• Annotations in Hibernate were extensively used.
• Most parts of the code developed were integrated using the Struts and Spring Framework.
• Involved in developing Hibernate mapping definitions for the required persistent layer.
• Implemented validations for both client side and Server side using the Struts2 Validator Framework.
• MVC design pattern and TDD was used in the application in order to make it more modular.
• Was responsible in generating Maven Script was faster and easier deployment.
• Configurations and Deployment was implemented using the IBMWebSphere Application using Maven Script.
• Deployed Web Services, JSP’s, Servlets, and Server side components in Web Logic Application Server.
• Worked on running integrated testing using the JUNIT andXML for building the data structures required for the Web Service.

Environment: s:Windows XP, Core Java (J2SE-Multithreading, IO), WSAD, TDD Methodology, AgileMethodology, J2EE-JSP, Servlets, UML (Rational Rose), Web logic Server, Tomcat, Struts2, Spring,Maven, CVS, JUnit, Log4j, EJB 2.0, XML, Oracle 10g, Stored Procedures, Eclipse, WebSphere v 6.

Confidential, Raleigh, NC

Application Developer J2EE

Responsibilities:

• Involvement with full cycle development of software which include Requirement Gathering, Design and Develop Applications.
• Actively participated in Project Designing using JSF Framework and SDLC phases of the MTM-Project.
• Developed user interface using JSF Prime faces with Java Beans, MDB’s Custom Tag Librariesand AJAXto speed the application.
• Used Java/J2EE Design Patterns like Session Façade, DAO Pattern, and MVC Pattern.
• Implemented controllers, services part.
• Used web services concepts like SOAP, WSDL, JAXB and JAXP to interact with other projects for sharing information.
• Developed JavaScript behavior code for user interaction.
• Created database program in SQL server to manipulate data accumulated by internet transactions.
• Wrote Servlets class to generate dynamic HTML pages.
• Developed Servlets and back-end Java classes using Web Sphere application server.
• Developed an API to write XML documents from a database.
• Performed usability testing for the application using JUnit Test.
• Created complex SQL and used JDBC connectivity to access the database.
• Involved in the design and coding of the data capture templates, presentation and component templates.
• Part of the team that designed, customized and implemented metadata search and database synchronization.
• Used Oracle as Database and used Toad for queries execution and also Involved in writing SQL scripts, PL SQL code for procedures and functions

Environment: Java, Web Sphere 3.5, EJB, JSF & Prime Faces 3.2, IBM-DB2, Servlets, JavaScript, JDBC, SQL, JUnit, Eclipse IDE. Apache Tomcat 6

Confidential

Java Developer

Responsibilities:

• Analyzed Business Requirements and Identified mapping documents required for system and functional testing efforts for all test scenarios.
• Performed Requirement Gathering & Analysis by actively soliciting, analyzing and negotiating customer requirements and prepared the requirements specification document for the application using Microsoft Word.
• Developed Use Case diagrams, business flow diagrams, Activity/State diagrams.
• Adopted J2EE design patterns like Service Locator, Session Facade and Singleton.
• Configuration of application using spring, Hibernate, DAO’s, Actions Classes, Java Server Pages.
• Configuring Hibernateand Tiles related XML files.
• Developed presentation layer using Java Server Faces (JSF) MVCframework.
• Used JSP, HTML and CSS, JQueryas view components in MVC.
• Extensively used Spring IOC for Dependency Injection and worked on Custom MVC Frameworks loosely based on Struts.
• Developed Servlets and Java Server Pages (JSP), to route the submittals to the EJB components and render-retrieved information using Session Facade, Service Locator (design pattern).
• Developed J2EE components on Eclipse IDE.
• Used JDBC to invoke Stored Procedures and also used JDBC for database connectivity to SQL.
• Deployed the applications on Web SphereApplication Server
• Developed Web servicesusing Restful and JSON.
• Used Oracle11g database for tables creation and involved in writing SQL queries using Joins and Stored Procedures.
• Used Toaddatabase tool to develop oracle quires.
• Writing complex SQL queries and reviewing SQL queries for the other team members.
• Developed JUnit Test Cases for Code unit test.
• Worked with configuration management groups for providing various deployment environments set up including System Integration testing, Quality Control testing etc.

Environment: Java/J2EE, SQL, Oracle 10g, JSP 2.0, EJB, AJAX, Java Script, Web Logic 8.0, HTML, JDBC 3.0, XML, JMS, log4j, JUnit, Servlets, MVC, My Eclipsed