SUMMARY:

• Over 8+ years of experience in Analysis, Architecture, Design, Development, Testing, Maintenance and User training of software application which includes over 5 years in Big Data, Hadoop and HDFS environment and 3 year experience in JAVA
• Experience on installing, configuring, and using Hadoop components like Hadoop Map Reduce (MR1), YARN (MR2), HDFS, Hive, Pig, Flume and Sqoop, Spark, Zookeeper.
• Experience in developing Map Reduce Programs using Apache Hadoop for analyzing the big data as per requirement.
• Hands on using Sqoop to import data into HDFS from RDBMS and vice - versa.
• Used different Hive Serde's like Regex Serde and HBase Serde.
• Experience in analyzing data using Hive, Pig Latin, and custom MR programs in Java.
• Hands on using job scheduling and monitoring tools like Oozie and Zookeeper
• Clear understanding on Hadoop architecture and various components such as HDFS, Job and Task Tracker, Name and Data Node, Secondary Name Node and Map Reduce programming.
• Hands on writing custom UDFs for extending Hive and Pig core functionality.
• Hands on dealing with log files to extract data and to copy into HDFS using flume.
• Wrote Hadoop Test Cases in Hadoop for checking Input and Outputs.
• Hands on integrating Hive and HBase.
• Experience in NOSQL databases: MongoDB, HBase, Cassandra
• Experience in Hadoop administration activities such as installation and configuration of clusters using Cloudera and HDP.
• Extensively worked with Cloudera Distribution Hadoop, CDH4.x, CDH 5.x
• Experience in JAVA, J2EE, Web Services, SOAP, HTML and XML related technologies demonstrating strong analytical and problem solving skills, computer proficiency and ability to follow through with projects from inception to completion.
• Extensive experience working in Oracle, DB2, SQL Server and My SQL database and Java Core concepts-OOPS, Multi-threading, Collections and IO.
• Hands on JAXWS, JSP, Servlets, Struts, Web Logic, Web Sphere, Hibernate, Spring, ANT, MAVEN, JBoss, JDBC, RMI, Java Script, Ajax, jQuery, UNIX, Linux, XML, and HTML.
• Developed applications using Java, RDBMS, and Linux shell scripting.
• Experience in complete project life cycle of Client Server and Web applications.
• Good understanding of Data Mining and Machine Learning techniques with R and Python.
• Excellent working experience in Scrum / Agile framework and Waterfall project execution methodologies.
• Experience in scripting to deploy monitors, checks and critical system admin functions automation
• Have good interpersonal, communication skills, strong problem solving skills, explore/adopt to new technologies with ease and a good team member and meets deadlines.
• Have the motivation to take independent responsibility and strong work ethic with desire to succeed and make significant contributions to the organization.

TECHNICAL SKILLS:

Big Data Ecosystem: Hadoop HDFS, Map Reduce, YARN, Hive, Pig, HBase, Zookeeper,Sqoop, Oozie, Kafka, Flume, Avro, Kibana, Spark, Splunk,Streaming, Storm

NoSQL Databases: HBase, Cassandra, MongoDB

Databases: Oracle 11g/10g, IBM DB2, SQL Server, Netezza, MySQL, MS -Access.

Tools: & IDE: Eclipse, NetBeans, Tableau, Toad, Rational Ross, Star UML

Programming Languages: C, C++, JAVA/J2EE, UNIX Shell Scripting, R, ScalaProgramming(RStudio) PIG, HiveQL

JAVA /J2EE Technologies: JSP, JSF, Servlets, JDBC, ODBC, XML, Swing, EJB, JSTL, JMS Frameworks: MVC, Spring3/2.5/2, Struts 2/1, Hibernate 3, ANT, MAVEN

Operating Systems: Windows, Linux- Ubuntu, RedHat, Solaris

Web Technologies: HTML, DHTML, CSS, XML, XSTL, JavaScript, JQuery

Web Services: Apache Tomcat, WebLogic, Wes Sphere, JBoss

SDLC Methodologies: Agile Scrum, SDLC activity, UML Design Patterns (Core Java/J2EE)

Network Protocols: HTTP, TCP/IP, UPD, DNS

WORK EXPERIENCE:

Confidential, Atlanta, GA

Sr. Hadoop Developer/Administrator

Responsibilities:

• Gathered the business requirements from the Business Partners and Subject Matter Experts.
• Involved with ingesting data received from various providers, on HDFS for big data operations.
• Accessed information through mobile networks and satellites from the equipment.
• Loaded and transformed large sets of structured, semi structured and un-structured data in various formats like text, zip, XML and JSON, AVRO.
• Imported data using Sqoop to load data from Oracle to HDFS on regular basis or from Oracle server to Hbase depending on requirements.
• Written Pig Latin scripts.
• Wrote Hive queries for data analysis to meet the business requirements.
• Created Hive tables and working on them using Hive QL.
• Loaded the data into Cassandra using CQL and carefully, generated the row key.
• Built Hive table on top of Cassandra so that it can point to the reporting tools like Tableau.
• Developed Scripts and Batch Job to schedule various Hadoop Programs.
• Participated in Admin role. Worked with hdfs-site.xml, hive-site.xml and other configurations during installation and working.
• Implemented advanced procedures like text analytics and processing using the in-memory computing capabilities like Spark.
• Enhanced and optimized product Spark code to aggregate, group and run data mining tasks using the Spark framework.
• Built web portal using JavaScript, it makes a REST API call to the elastic search and gets the row key.
• Used Kibana, which is an open source based browser analytics and search dashboard for Elastic Search.
• Monitored the Hadoop Cluster using Cloudera Manager.
• Utilized Agile Scrum Methodology to help manage and organize a team of 4 developers with regular code review sessions.
• Weekly meetings with technical collaborators and active participation in code review sessions with senior and junior developers.

Environment: HDFS, Pig, Hive, Cassandra, Sqoop, Apache Spark, Tableau, Talend, Elastic Search, Kibana, SFTP, Cloudera CDH 5.x

Hadoop Developer

Responsibilities:

• Teamed up with Data Architects and Source System Analysts to map the data from various sources to target attributes.
• Knowledge on integration of various data sources to a Hadoop Data Lake in a Communication model.
• Imported structured data from file systems using file copy component and RDBMS through Apache Sqoop import.
• Imported streaming data using Apache Storm and Apache Kafka into HBase and designed Hive tables on top.
• Implemented Pre-Preparation layer using Data profiling as specified in the Source to Target Mapping document.
• Handled various special character issues during ingestion using Apache Sqoop and Pig cleansing techniques.
• Wrote workflows which include data cleansing Pig actions and hive actions.
• Developed Java UDF's for Date conversions and to generate MD5 checksum value.
• Implemented Apache Spark data processing project to handle data from RDBMS and streaming sources.
• Designed batch processing jobs using Apache Spark to increase speeds by ten-fold compared to that of MR jobs.
• Designed custom Spark REPL application to handle similar datasets.
• Reduced Code redundancy by identifying and parameterizing frequently used values in the shell script.
• Knowledge on implementing various SCD user requirements.
• Optimized Pig joins to increase performance.
• Scheduled the jobs using Apache Oozie to run during off-peak hours for load management on Cluster.
• Prepared and scheduled dispatch jobs which include Teradata BTEQ scripts to load the processed data into Teradata.
• Used Resource Manager for monitoring Job status and for debugging the Mapper/Reducer failures.

Environment: Hadoop, Map Reduce, HDFS, Hive, Phoenix, Pig, Sqoop, Oozie, Solr, Spark, SQL, Java (jdk 1.6), Tableau, Eclipse, Jenkins, Nexus, AVRO, TIDAL.

Hadoop Developer

Responsibilities:

• Responsible for building scalable distributed data solutions using Hadoop.
• Hadoop clusters are Setup and benchmarked for internal use.
• Hands on extracting data from different databases and to copy into HDFS file system using Sqoop.
• Hands on creating different applications in social networking websites and obtaining access data from them.
• Developed simple to complex Map Reduce jobs using Hive and Pig for analyzing the data.
• Used different Serde's for converting JSON data into pipe separated data.
• Implemented some business logics by writing UDFs in Java and used various UDFs from Piggybanks and other sources to get some results from the data.
• Optimized jobs to use HDFS efficiently by using various compression mechanisms.
• Used Oozie workflow engine to run multiple Hive and Pig jobs.
• Hands on exporting the analyzed data into relational databases using Sqoop for visualization and to generate reports for the BI team.
• Involved in installing and configuring Hive, Pig, Sqoop, Flume and Oozie on the Hadoop cluster.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
• Continuously monitored and managed the Hadoop Cluster using Cloudera Manager.

Environment: Hadoop, Cloudera Manager, HDFS, Hive, Pig, Sqoop, Oozie, SQL, Eclipse, SQL Server 2008, Ubuntu 13.04.

JAVA Developer

Responsibilities:

• Involved in Analysis of the requirements.
• Prepared the High and Low level design document.
• Used spring‘s MVC framework at the front end of the application.
• Used Agile Software Development for developing the application and developed Use case diagrams, Object diagrams, Class diagrams, and Sequence diagrams using UML.
• Implemented Connection pool object for database connectivity.
• Involved in development of DAO layer for using Hibernate.
• Involved in development of Procedures, functions and triggers in the database.
• Implemented various J2EE design patterns such as Session Façade, Business Delegate, Value Object, Data Access Object, etc.
• Used XML parsers to parse incoming data and populating the database with the data
• Designed the GUI screens using Struts and Configured log4j to debug the Application.
• Involved in the development of test cases for the testing phase.
• Used Assertions for writing the assert statements in the Junit test cases.
• Used Test Driven Development in writing the test cases for developing the Java modules.
• Used JSF standard validations &JavaScript for client-side validations.
• Used hibernate for Data persistence in back end.
• Packaged and deployed the application in Web sphere server in different environments.
• Performed End to end integration testing of online scenarios and unit testing using JUnit Testing Framework.

Environment: Java, Windows XP, RAD 7.0.1, Apache Tomcat, JSP, JSF, Java Script, Struts, SQL, JDBC, UML, CSS, Hibernate, JUnit.