SUMMARY

• Around 10+ years of Professional experience in IT Industry, involved in Developing, Implementing, Configuring Hadoop ecosystem components on Linux environment, Development and maintenance of various applications using Java, J2EE, developing strategic methods for deploying Big data technologies to efficiently solve Big Data processing requirement.
• 4 years of experience as Hadoop Developer with sound knowledge in Hadoop ecosystem technologies.
• Hands on experience in Hadoop eco system components such as HDFS, MapReduce, Yarn, Pig, Hive, Hbase, Oozie, Zookeeper, Sqoop, Flume, Impala, Kafka and Strom.
• Excellent Programming skills at a higher level of abstraction using Scala and Spark.
• Good understanding in processing of real - time data using Spark.
• Hands on experience in Importing and exporting data from different databases like MySQL, Oracle, Teradata into HDFS using Sqoop
• Strong experience working with real time streaming applications and batch style large scale distributed computing applications using tools like Spark Streaming, Kafka, Storm, Flume, MapReduce, Hive.
• Experience working with Cassandra and NoSQL database including MongoDB and Hbase.
• Experience with messaging & complex event processing systems such as Kafka and Storm.
• Also, have experience in Using Kafka and Storm to stream Real time data into our Cluster(HDFS).
• Managing and scheduling batch Jobs on a Hadoop Cluster using Oozie.
• Experience in managing and reviewing Hadoop Log files.
• Used Zookeeper to provide coordination services to the cluster.
• Familiarity with NoSQL databases including HBase, MongoDB.
• Experienced using Sqoop to import data into HDFS from RDBMS and vice-versa.
• Experience and understanding in Spark and Storm.
• Hands on dealing with log files to extract data and to copy into HDFS using flume.
• Experience in analyzing data using Hive, Pig Latin, and custom MR programs in Java.
• Experience in designing and coding web applications using Core Java & Web Technologies- JSP, Servlets and JDBC, full Understanding of utilizing J2EE technology Stack, including Java related frameworks like Spring, ORM Frameworks(Hibernate).
• Experienced in the use of agile approaches, including Extreme Programming (XP Programming Techniques), Test-Driven Development (TDD Methodologies) and Scrum.
• Experience in designing the User Interfaces using HTML, CSS, JavaScript and JSP.
• Developed web application in open source java framework Spring. Utilized Spring MVC framework.sc
• Experienced front-end development using EXT-JS, JQuery, JavaScript, HTML, Ajax and CSS.
• Developed RESTful Web Services using Spring Rest and Jersey framework.

TECHNICAL SKILLS

Big Data: Apache Hadoop, HDFS, Map Reduce, Hive, PIG, OOZIE, SQOOP, Spark, Hortonworks Data Platform, Cloudera Manager, MapR and EMR

Database: MYSQL, Oracle, SQL Server, Hbase

IDE’s: Eclipse, Net beans

Languages: C, Java, PIG LATIN, UNIX/LINUX Shell Scripting, Python

Scripting Languages: HTML, CSS, Java Script, DHTML, XML, JQuery

Web Technologies: HTML, XML, Java Script, JQuery

Web/Application Servers: Apache Tomcat, Web Logic

PROFESSIONAL EXPERIENCE

Confidential, Greenville, SC

Sr. Hadoop/Big Data Developer

Responsibilities:

• Worked on Hadoop cluster and data querying tools Hive to store and retrieve data while developing, applications involved in complete Software Development Life Cycle(SDLC).
• Reviewing and managing Hadoop log files by consolidating logs from multiple machines using flume.
• Developed Oozie workflow for scheduling ETL process and Hive Scripts
• Involved in teams to analyze the Anomaly detection and ratings of data.
• Implemented custom input format and record reader to read XML input efficiently using SAX parser.
• Involved in Requirement Analysis, Design, and Development.
• Developed multiple MapReduce jobs in java for data cleaning and Preprocessing.
• Created MapReduce jobs using Pig Latin and Hive Queries Also Core Java.
• Developed Pig Latin scripts to do operations of sorting, joining and filtering enterprise data.
• Used Kafka and Storm for real time data ingestion and processing.
• Implemented test scripts to support test driven development and integration.
• Involved in converting Hive/Sql queries into Spark transformations using Spark RDD’s.
• Worked on SPARK to retrieve data by using SCALA.
• Developed Spark Scripts by using Scala shell commands as per the requirement.
• Developed Scala Scripts, UDF’s using both Data frames/SQL and RDD/MapReduce in spark for Data Aggregation, queries and writing data back into RDBMS through Sqoop.
• Developed ETL jobs using Spark-Scala to migrate data from Oracle to new hive tables.
• Worked on executors on each assigned cluster nodes-work processes to run computations and store data using spark context.
• Developed UDF’s using both Data Frames/Sql and RDD in Spark for Data Aggregation queries and reverting into OLTP through Sqoop.
• Using CSV Excel Storage to parse with different delimiters in PIG.
• Installed and monitored Hadoop ecosystems tools on multiple operating systems like Ubuntu, CentOS.
• Modified reports and Talend ETL jobs based on the feedback from QA testers and Users in development and staging environments.
• Responsible for designing and implementing ETL process using Talend to load data from Worked extensively with Sqoop for importing and exporting the data from HDFS to Relational Database systems/mainframe and vice-versa. Loading data into HDFS.
• Involved in setting QA environment by implementing pig and Sqoop scripts.
• Involved in loading data from Linux file systems, servers, java web services using Kafka producers and consumers.
• Involved in developing code to write canonical model JSON records from numerous input sources to Kafka Queues.
• Collected the logs data from web servers and integrated in to HDFS using Flume.
• Developed UNIX shell scripts for creating the reports from Hive data.
• Manipulate, serialize, model data in multiple forms like JSON, XML.
• Prepared avro schema files for generating Hive tables
• Experienced in working with key applications team in installing Hadoop updates, upgrades based on requirement.

Environment: Hadoop Map Reduce 2 (yarn), Hortonworks Distribution (HDP), HDFS, Core Java, PIG, Hive, Flume, Cassandra, Eclipse, Java, Sqoop, Spark, Spark Sql, Linux shell scripting.

Confidential, CA

Hadoop/Big Data Developer

Responsibilities:

• Developed data pipeline using Flume, Sqoop, Pig and Java map reduce to ingest customer behavioral data and financial histories into HDFS for analysis.
• Worked on importing and exporting data from Oracle and DB2 into HDFS and HIVE using Sqoop for analysis, visualization and to generate reports.
• Developed multiple MapReduce jobs in java for data cleaning.
• Used various technologies such as Sqoop, Pig, Hive, MapReduce and Shell Scripts.
• Developed Hive UDF to parse the staged raw data to get the Hit Times of the claims from a specific branch for a particular insurance type code.
• Schedule these jobs with workflow engine like Oozie. Actions can be performed both sequentially and parallely using Oozie.
• Develop and maintain several batch jobs to run automatically depending on business requirements. Built wrapper shell scripts to hold this Oozie workflow.
• Involved in collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
• Involved in creating Hadoop streaming jobs using Python.
• Used Tableau for Data Visualization of queries in the Hive Summary tables.
• Used Ganglia to Monitor and Nagios to send alerts about the cluster around the clock
• Provided ad-hoc queries and data metrics to the Business Users using Hive, Pig.
• Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS.
• Used Pig as ETL tool to do transformations, event joins and some pre-aggregations before storing the data onto HDFS.
• Worked on MapReduce Joins in querying multiple semi-structured data as per analytic needs.
• Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
• Created many Java UDF and UDAFs in hive for functions that were not preexisting in Hive like the rank, Csum, etc.
• Automate provisioning of Azure infrastructure with PowerShell, AzureResource Manager and DSC.
• Used Hive and created Hive tables and involved in data loading and writing Hive UDFs.
• Developed POC for Apache Kafka.
• Worked on various performance optimizations like using distributed cache for small datasets, partition and bucketing in hive, doing map side joins etc..
• Storing and loading the data from HDFS to Amazon S3 and backing up the Namespace data into NFS Filers.
• Created concurrent access for hive tables with shared and exclusive locking that can be enabled in hive with the help of Zookeeper implementation in the cluster.
• Wrote the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
• Wrote shell scripts for rolling day-to-day processes and it is automated.
• Monitor Apache Hadoop cluster connectivity and security.
• Involved in story-driven agile development methodology and actively participated in daily scrum meetings.

Environment: Hadoop, AWS, MapReduce, Hive, HDFS, PIG, Sqoop, Oozie, Cloudera, Flume, HBase, Zookeeper, CDH3, MongoDB, Cassandra, Oracle, NoSQL and Unix/Linux, Kafka, Amazon web services.

Confidential

Java Developer

Responsibilities:

• Developed the application using Struts Framework that leverages classical Model View Layer (MVC) architecture UML diagrams like use cases, class diagrams, interaction diagrams, and activity diagrams were used.
• Participated in requirement gathering and converting the requirements into technical specifications.
• Extensively worked on User Interface for few modules using JSPs, JavaScript and Ajax.
• Created Business Logic using Servlets, Session beans and deployed them on Web logic server.
• Wrote complex SQL queries and stored procedures.
• Developed the XML Schema and Amazon Web services for the data maintenance and structures.
• Worked on analyzing, writing Hadoop MapReduce jobs using Java API, Pig and hive.
• Selecting the appropriate AWS service based upon data,compute, system requirements.
• Implemented the Web Service client for the login authentication, credit reports and applicant information using Apache Axis 2 Web Service.
• Worked on installing cluster commissioning decommissioning of data node name node recovery configuration. Got good experience with NOSQL database like MongoDB.
• Designed the logical and physical data model, generated DDL scripts, and wrote DML scripts for Oracle 9i database.
• Used Hibernate ORM framework with spring framework for data persistence and transaction management.
• Used struts validation framework for form level validation.
• Wrote test cases in JUnit for unit testing of classes.
• Involved in templates and screens in HTML and JavaScript.
• Involved in integrating Web Services using WSDL and UDDI.
• Created HBase tables to load large sets of data coming from UNIX and NoSQL.
• Built and deployed Java applications into multiple Unix based environments and produced both unit and functional test results along with release notes.

Environment: JDK 1.5, J2EE 1.4, Struts 1.3, JSP, Servlets 2.5, Web Sphere 6.1, HTML, XML, ANT 1.6, JavaScript, Node.js, JUnit 3.8, HDFS, MongoDB, Hive, HBase UNIX, AWS

Confidential

Java Developer

Responsibilities:

• Played an active role in the team by interacting with welfare business analyst/program specialists and converted business requirements into system requirements.
• Developed and deployed UI layer logics of sites using JSP.
• Struts (MVC) is used for implementation of business model logic.
• Worked with Struts MVC objects like Action Servlets, Controllers, and validates, Web Application Context, Handler Mapping, Message Resource Bundles and JNDI for look-up for J2EE components.
• Developed dynamic JSP pages with Struts.
• Developed the XML data object to generate the PDF documents and other reports.
• Used Hibernate, DAO, and JDBC for data retrieval and medications from database.
• Messaging and interaction of Web Services is done using SOAP and REST
• Developed JUnit Test cases for Unit Test cases and as well as System and User test scenarios
• Involved in Unit Testing, User Acceptance Testing and Bug Fixing.

Environment: J2EE, JDBC, Java 1.4, Servlets, JSP, Struts, Hibernate, Web services, RESTful services, SOAP, WSDL, Design Patterns, MVC, HTML, JavaScript 1.2, Web Logic 8.0, XML, Junit, Oracle 10g, My Eclipse.

Confidential

Java Developer

Responsibilities:

• Designed Entity relationship diagram (ERD) using the Erwin tool.
• Involved in Data modeling and data base design of the all the modules.
• Configured Log4j for logging, debugging purpose in the application.
• Quality Control Activities such as defect tracking, fixing using JIRA.
• Involved in Developing UML artifacts such as Class Diagrams, Activity Diagrams using MS Visio.
• Developed Menu-driven Web Portal using HTML, Java script, CSS, Servlets/JSP.
• Developed “Incident management” module using Design patterns such as Composite, Factory patterns and using EJB transaction management.
• Trouble shooting, and tracking defects towards closure.
• Configure JDBC Data Sources and Connection pooling in Web logic application Server.
• Configured and maintained JMS Queues, Topics in Web logic application Server.
• Involved in Change Request management, impact analysis and implementation of modules.
• Responsible for maintaining the code repository in PVCS.
• Developed ANT scripts to build and deploy the application.
• Involved in SIT and UAT phases of the Application.
• Prepared SOAPUI unit test cases, for each service.
• Deployed the application on Solaris box after completion of the enhancement.

Environment: JSP, JavaScript, Servlets, XML, HTML, Hibernate, JQuery