SUMMARY:

• Around 8+ years of Experience in IT Industry in Application Development and Data Analytics using various languages and 3 years of experience in Big Data Hadoop Eco - System, developing java code and Strong in Design, Requirement gathering, Analysis, development, implementation and support of applications in the role of Java Developer and Big Data Hadoop Developer.
• Proficiency in Big Data Practices and Technologies like HDFS, YARN, MapReduce, Hive, PIG, HBase, SQOOP, OOZIE, ZOOKEEPER, Flume, Kafka, Impala, Spark.
• Excellent Hands on Experience in developing Hadoop Architecture in Windows and Linux platforms.
• Capable of processing large sets of structured, semi-structured and unstructured data and supporting systems application architecture.
• Experience in writing MapReduce programs with custom logics based on the requirement.
• Experience in writing custom UDFs in hive based on the user requirement.
• Strong Knowledge of using PIG and Hive for processing and analyzing large volumes of data.
• Expert in extending core functionality of Hive and PIG by writing the Custom UDF’s using Java, Python based on user requirement.
• Experienced in Writing Hive queries for data analysis and to process the data for visualization using Tableau and Splunk.
• Expertise in importing and exporting data using SQOOP from HDFS to various Database Systems (RDBMS, Data Ware Houses, Data lake)and vice-versa .
• Hand on Experience on the SPARK SQL, and SPARK Streaming modules of Spark extensively and used Python and Scala using pyspark API’s and Spark to write code for all Spark Use cases.
• Experience in ETL tools like Talend and in-depth knowledge on Working of Data WareHouses.
• Capable of creating Real time data streaming Solutions and Batch style large scale distributed computing applications using Apache Spark, Spark Streaming, Kafka and Flume.
• Knowledge and experience in job work-flow scheduling and monitoring tools like Autosys, Control M, Oozie and Zookeeper.
• Proficient in writing in shell and Perl scripts and proficient in Unix/Linux Commands.
• Experience in storing, processing unstructured data using NoSQL databases like HBase, MongoDB, and Cassandra.
• Good Knowledge on AWS Concepts like EC2, S3, and EMR.
• Experience in performing Unit Test, Regression Test, Functional test and Integration Test and experienced in using MRUnit for Hadoop Testing.
• Experience in Object Oriented Analysis and Design ( OOAD ) and development of software using Java, Scala, and Python in LINUX/UNIX platform.
• Extensive Experience in middle-tier development using J2EE Technologies and Frameworks like struts, Restful Web services, SOA, spring, Hibernate and Application servers like glassfish and Tomcat.
• Proficient in working with version controls Like GIT, SVN and with Bug Tracking Tool like JIRA.
• Proficient in SQL and PL/QL Using Oracle, DB2, Teradata, MySQL and SQL Worked on Agile Environment and Waterfall Methodology.
• Good experience with Build Tools like Maven, Ant and CI and CD tools Like Jenkins.
• Experience in managing and reviewing Hadoop Log Files.
• A self-motivated professional possessing good technical, Initiating and Problem-solving skills and has proven to be a good team Player and able to work single.

TECHNICAL SKILLS:

Big Data Eco Components: HDFS, MapReduce, Yarn, Hive, SQOOP, Pig, HBASE, Zookeeper, Oozie, Impala, Flume, Kafka, Spark.

Programming Languages: Java, C, C++, Python, PL/SQL,Scala, shell scripting

J2EE Components: JDBC, JSP, Servlets, EJBs, and Design Patterns.

Frameworks: Spring, Hibernate, Struts, Restful and SOAP

Databases: Oracle MySQL, SQL Server, Cassandra.

Application Servers: Tomcat, Glass Fish, Web Sphere and Web Logic

Build Tools and Central repositories: Maven, Ant, GIT and SVN.

IDEs: Eclipse, NetBeans, Sublime

Methodology: Agile and Waterfall

Environment: Windows and Linux .

PROFESSIONAL EXPERIENCE:

Confidential, California

Hadoop Spark Developer

Responsibilities:

• Responsible for building scalable distributed data solutions on Hadoop Cluster of size 150 nodes.
• Performed data graduation from traditional data warehouse (Teradata) to Hadoop Data Lake
• Implemented data ingestion from multiple sources like Teradata, Oracle into Hadoop using SQOOP.
• Involved in collecting and aggregating large amounts of log data using Apache Flume and staging the data in HBase/HDFS for further Analysis.
• Collected the logs data from web servers and integrated into HBase using Flume.
• Developed Map Reduce programs that filter un-necessary records and find out unique records based on different criteria.
• Responsible for performing extensive data validation using Hive.
• Experienced in implementing POC's to migrate iterative map reduce programs into Spark transformations using Scala.
• Worked on Sequence files, RC files, Map side joins, bucketing, partitioning for Hive performance enhancement and storage improvement.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Good experience in Hive partitioning, bucketing and perform different types of joins on Hive tables and implementing Hive SerDe's like REGEX, JSON and Avro.
• Moving the data from Oracle, MS SQL Server in to HDFS using Sqoop and importing various formats of flat files in to HDFS.
• Migrated complex Map reduce programs into Spark RDD transformations, actions.
• Improving the performance and optimization of existing algorithms in Hadoop using Spark context, Spark-SQL and Spark YARN using Scala.
• Implemented Spark Core in Scala to process data in memory.
• Performed job functions using Spark API's in Scala for real time analysis and for fast querying purposes.
• Importing and exporting data into HDFS, Hive and HBase using SQOOP.
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Python and Scala.
• Optimized MapReduce jobs to use HDFS efficiently using various compression Mechanisms.
• Worked on migrating PIG scripts and MapReduce programs to Spark Data frames API and Spark SQL to improve performance.
• Used Kafka, Flume for building robust and fault tolerant data Ingestion pipeline for transporting streaming web log data into HDFS.
• Developed a data pipeline using Kafka and storm to store data in HDFS.
• Used Maven to Build and Deploy Jar’s for MapReduce, Hive and PIG UDF’s.
• Developed Hive QL to process the data and to generate the Reports for visualizing.

Environment: Hadoop, MapR, MapReduce, HDFS, Hive, Pig, Zookeeper, pySpark, SparkSQL, Spark Streaming, Scala, Python, impala, EDW, Maven, Jenkins, Sqoop, Oozie, Kafka, Teradata.

Hadoop Developer

Responsibilities:

• Processed data into HDFS by developing solutions, analyzed the data using MapReduce, Hive and produce summary results from Hadoop to downstream systems.
• Developed multiple MapReduce jobs in Java for data cleaning and preprocessing.
• Implemented flume (Multiplexing) to stream data from upstream pipes in to HDFS.
• Used SQOOP widely to import data from various RDBMS (DB2) into HDFSand to Move data between MongoDB and HDFS.
• Involved in Hive partitioning, Bucketing, and performing different types of joins on Hive table and implementing serde’s like RegEx.
• Responsible for writing Hive Queries for analyzing data in Hive warehouse using Hive Query Language (HQL) and Hive UDF’s in python.
• Created scalable and High Performances Rest web services for data tracking.
• Applied Hive quires to perform analysis of vast data on HBASE using Storage Handler to meet the business requirements.
• Involve in collecting, aggregating and moving data from servers to HDFS using Apache Flume.
• Worked on NoSQL databases like HBase, integration with written storm topology to accept inputs from Kafka producer.
• Planned, Implemented and Managed Splunk for log Management and analytics.
• Implemented Scala jobs to integrate the real-time data coming from various queues messaging to parse it.
• Used Pig as ETL tool to do Transformations, even joins and some pre-aggregations.
• Load and Transform Large Sets of Structured and semi-structured data
• Developing Scripts and Batch Job to schedule a bundle (group of coordinators) which consists of various Hadoop Programs using OOZIE.
• Exported the analyzed data to the relational databases using SQOOP for visualization and to generate reports for the BI team.
• Involved in writing Unix/Linux Shell scripting for scheduling jobs and for writing Pig scripts and Hive QL.
• Cluster co-ordination services through Zookeeper.
• Involved in upgrading clusters to Cloudera Distributed versions.
• Designed, documented operational problems by following standards and procedures using a software reporting tool JIRA.
• Automated Build upon check in with Jenkins CI (continuous Integration)
• Implemented test Scripts to support test driven development and continuous integration.
• Assisted in Cluster maintenance, cluster monitoring, adding and removing cluster nodes and Troubleshooting.

Environment: Hadoop, MapReduce, HDFS,SQOOP, Flume, Kafka, LINUX, OOZIE, Python, Splunk, Pig, Scala, ETL, MySQL, JIRA, Hive, Jenkins, HBASE, DB2, MongoDB, Cloudera Hadoop Cluster.

Hadoop Admin/Developer

Responsibilities:

• Responsible for building scalable distributed data solutions using Hadoop.
• Involved in start to end process of Hadoop Cluster setup including installation, configuration and monitoring the Hadoop Cluster.
• Administered Cluster Maintenance, commissioning and decommissioning data nodes, Cluster monitoring troubleshooting.
• Performed Adding/Removing nodes to an existing Hadoop cluster.
• Implemented Backup configurations and recoveries from a Name Node failure.
• Monitored systems and services, architecture design and implementation of Hadoop Deployment, configuration management backup and disaster recovery systems and procedures.
• Implemented multiple MapReduce programs in Java for Data Analysis.
• Performed performance tuning and troubleshooting of MapReduce jobs by analyzing and reviewing Hadoop log files.
• Developed MapReduce programs to parse the raw data, populate staging tables and store the refined data in partitioned tables in the Enterprise Data Ware House (EDW).
• Created Hive Queries for the Market analysts to analyze the emerging data and comparing it with fresh data with EDW references tables.
• Collected the logs from the physical machines and the OpenStack controller and integrated into HDFS using Flume.
• Designed and presented plan for POC on impala.
• Involved in migrating HiveQL into Impala to minimize query response time.
• Implemented Avro and parquet data formats for apache Hive computations to handle custom business requirements.
• Worked on Sequence files, ORC, RC files, Map side joins, bucketing, partitioning for Hive.
• Performance enhancement and storage improvement.
• Performed extensive Data Mining applications using HIVE.
• SQOOP jobs, PIG and Hive scripts were created for data ingestion from relational databases (MySQL) to compare with historical data.
• Worked on Storm real time processing bolts which save data to SOLR and HBase.
• POC for Enabling members and suspect search using SOLR.
• Used Pig as ETL tool to do transformations, event joins, filter and some pre-aggregations.
• Used Visualization tools such as Power view for excel Tableau for visualizing and generating reports.
• Maintained and managed code using GIT and used JIRA for BUG Tracking
• Performed complex Linux administrative activities as we created, maintained and updated Linux shell scripts
• Implemented test scripts to support test driven development and continuous integration.
• Involved in story-driven agile development methodology and actively participated in daily scrum meetings.

Environment: Hadoop, MapReduce, HDFS, Pig, Hive, Sqoop, STORM, SOLR, Flume, JIRA, Control M, Java, Linux, Maven, Zookeeper, EDW, GIT, ETL, Tableau, Cloudera, MySQL.

Java Hadoop Developer

Responsibilities:

• Involved in gathering requirements and converting the requirement into technical specifications.
• Implemented various OOP concepts and design patterns such as IOC (DI), Singleton, DAO, and prototype.
• Designing and implementing Spring UI Layer for the Application using the Spring MVC and Java Scripts.
• Involved in Developing Restful web services, Deployment configurations and testing Using Jersey.
• Involved in setting up and Monitoring the Hadoop Cluster along with Administrator.
• Responsible for managing data coming from different sources
• Involved in HDFS maintenance and loading of structured and unstructured data into HDFS.
• Developed and supported MapReduce programs to perform data filtering for unstructured data and jobs that’s are running on Hadoop Clustering.
• Implemented the import and export of Data using SQOOP between MySQL to HDFS on regular basis.
• Used Flume to load the data from different sources like file systems, servers into HDFS.
• Created partitioned Hive tables and wrote Hive queries for Data Analysis to meet the Business requirements.
• Designed and Developed UDF’s to extend the functionality in both PIG and Hive.
• Developed scripts and batch jobs to schedule various Hadoop Programs.
• Developed Hibernate with Spring Integration as the data abstraction to interact with the database.
• Configured Apache HTTP server and Apache Tomcat Server.
• Designed and Maintained Control M work flow to manage the flow of jobs in the cluster.
• Involved in Unit Testing and Developed Junit Test cases for unit testing and used various mock up frameworks like mock it Rest client UI.
• Actively updated the upper management with Daily updates on the progress of projects that include the classifications levels that were achieved on the data.

Environment: Java, Hadoop, HDFS, MapReduce, PIG, Hive, SQOOP, Control M, Linux, MySQL, J2EE, spring, Spring MVC, Hibernate, SQL, Restful Web Services, Apache Tomcat, Junit, Maven, HTML, JSP.

Java developer

Responsibilities:

• Extensively worked on Struts Framework.
• Associated in designing application using MVC design pattern.
• Developed front-end user interface modules by using HTML, XML, Java AWT, and Swing.
• Front-end validations of user requests carried out using Java Script.
• Designed and developed the interacting JSPs and Servlets for modules like User Authentication and Summary Display.
• Used Jconsole for the memory management.
• Developing Action, Action Form, Front Controller, Singleton Classes, and Transfer Objects (TO), Business Delegates (BD), Session Façade, Data Access Objects (DAO) and business validators.
• Analyzed, designed, implemented and integrated product in existing application.
• Wrote Jboss Quartz to schedule jobs.
• Communicated with the other components using JMS within the system.
• Designed and Developed Web Services implemented SOA architecture using SOAP and XML for the module and published (exposed) the Web Services.
• Used JDBC to connect the J2EEserver with the relational database.
• User input validations done using JavaScript and developed use cases using UML.
• Extreme programming methodologies for replacing the existing code and testing in J2EE environment.
• Developed web pages using HTML5, DOM, CSS3, JSON, JavaScript, JQuery and AJAX.
• Implemented applications using Bootstrap framework.
• Worked on developing internal customer service representative (CSR's) tools.
• Redesigned the service plan page to display dynamically service products based on user selection.
• Created functional requirements document for rental car industry with telematics system capability and use cases integrating Verizon managed certificate services into the Verizon M2M Management Center.
• Developed java classes for business layer.
• Developed database objects like tables, views, stored procedures, indexes.
• Involved in testing and fixing the bugs.

Environment: Java, J2EE, JSP, Servlets, Struts, HTML, Maven, Java Script, JDBC, Oracle (PL/SQL), DAO, Tomcat, JUnit, Eclipse.

Java Developer

Responsibilities:

• Extensively involved with the end-users in Requirements analysis, Object Oriented Analysis and Design (OOAD) for designing the database and in writing detailed technical and business specification documents using UML.
• Involved on the server side coding of Student Verification, Book Submission request, and Student details connectivity with the Front-end side using Servlets and JSP.
• Actively involved with the team members in the implementation of various Library modules that require database interactions using Triggers, Procedures using PL / SQL for Oracle.
• Actively involved in the Documentation and development of user and programmer manuals.

Environment: Windows, JSP, Servlets, HTML, Java1.6, J2EE, JDBC, PL/SQL, JWS 2.0, IIS, Oracle 10g.