SUMMARY:

• Over 9+ years of working experience as a Sr. Big Data Developer in designed and developed various applications on Big data and Java open - source technologies.
• Strong development skills in Hadoop, HDFS, Map Reduce, Hive, Sqoop, HBase with solid understanding of Hadoop internals.
• Hands on experience on Hadoop /Big Data related technology experience in Storage, Querying, Processing and analysis of data.
• Leverage strong skills in developing applications involving Big Data technologies like Hadoop, Map Reduce, Yarn, Flume, Hive, Pig, Sqoop, H Base, PIVotal, Cloudera, Map R, Avro, Spark and Scala.
• Extensively worked on major components of Hadoop Ecosystem like HDFS, HBase, Hive, Sqoop, PIG, and MapReduce.
• Develop various scripts, numerous batch jobs to schedule various Hadoop programs.
• Experience in analyzing data using Hive QL, and custom MapReduce programs in Java.
• Hands on experience in importing and exporting data from different databases like Oracle, MySQL, into HDFS and Hive using Sqoop.
• Good knowledge of NoSQL databases like Mongo DB, Cassandra and HBase.
• Expertise in writing Hadoop Jobs to analyze data using MapReduce, Hive, Pig and SOLR, Splunk.
• Experience in Programming and Development of java modules for an existing web portal based in Java using technologies like JSP, Servlets, JavaScript and HTML, SOA with MVC architecture.
• Strong Programming Skills in designing and implementing of multi-tier applications using Java, J2EE, JDBC, JSP, JSTL, HTML, CSS, JSF, Struts, JavaScript, Servlets, POJO, EJB.
• Extensive experience in SOA-based solutions - Web Services, Web API, WCF, SOAP including Restful APIs services
• Good Knowledge in Amazon Web Service (AWS) concepts like EMR and EC2 web services which provides fast and efficient processing of Teradata Big Data Analytics.
• Experienced in collection of Log Data and JSON data into HDFS using Flume and processed the data using Hive/Pig.
• Expertise in developing a simple web based application using J2EE technologies like JSP, Servlets, and JDBC.
• Experience working on EC2 (Elastic Compute Cloud) cluster instances, setup data buckets on S3 (Simple Storage Service), set EMR (Elastic MapReduce).
• Work Extensively in Core Java, Struts, JSF, Spring, Hibernate, Servlets, JSP and Hands-on experience with PL/SQL, XML and SOAP.
• In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, NameNode, DataNode.
• Well versed working with Relational Database Management Systems as Oracle, MS SQL, MySQL Server
• Hands on experience in advanced Big-Data technologies like Spark Ecosystem (Spark SQL, MLlib, SparkR and Spark Streaming), Kafka and Predictive analytics
• Knowledge of the software Development Life Cycle (SDLC), Agile and Waterfall Methodologies.
• Good knowledge of NoSQL databases such as HBase, MongoDB and Cassandra.
• Experience in working with Eclipse IDE, Net Beans, and Rational Application Developer.
• Experience includes Requirements Gathering, Design, Development, Integration, Documentation, Testing and Build.

PROFESSIONAL EXPERIENCE:

Confidential - San Antonio, TX

Sr. Big Data Developer

Responsibilities:

• As a Sr. Big Data Developer worked on Hadoop eco-systems including Hive, MongoDB, Zookeeper, Spark Streaming with MapR distribution.
• Developed Big Data solutions focused on pattern matching and predictive modeling.
• Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, HBase database and Sqoop.
• Involved in Agile methodologies, daily scrum meetings, spring planning.
• Implemented Security in Web Applications using Azure and deployed Web Applications to Azure.
• Involved in writing Spark applications using Scala to perform various data cleansing, validation, transformation and summarization activities according to the requirement.
• Load the data into Spark RDD and Perform in-memory data computation to generate the output as per the requirements.
• Used Kibana, which is an open source based browser analytics and search dashboard for Elastic Search.
• Developed Spark code and Spark-SQL/Streaming for faster testing and processing of data.
• Used Java Persistence API (JPA) framework for object relational mapping which is based on POJO Classes.
• Upgraded the Hadoop Cluster from CDH3 to CDH4, setting up High Availability Cluster and integrating Hive with existing applications.
• Designed & Developed a Flattened View (Merge and Flattened dataset) de-normalizing several Datasets in Hive/HDFS which consists of key attributes consumed by Business and other down streams.
• Worked on NoSQL support enterprise production and loading data into HBase using Impala and Sqoop.
• Performed multiple MapReduce jobs in Pig and Hive for data cleaning and pre-processing.
• Build Hadoop solutions for big data problems using MR1 and MR2 in YARN.
• Handled importing of data from various data sources, performed transformations using Hive, PIG, and loaded data into HDFS.
• Involved in identifying job dependencies to design workflow for Oozie & YARN resource management.
• Designed solution for various system components using Microsoft Azure.
• Worked on data using Sqoop from HDFS to Relational Database Systems and vice-versa. Maintaining and troubleshooting
• Exploring with Spark to improve the performance and optimization of the existing algorithms in Hadoop using Spark context, Spark-SQL, Data Frame, pair RDD's.
• Created Hive Tables, loaded claims data from Oracle using Sqoop and loaded the processed data into target database.
• Involved in PL/SQL query optimization to reduce the overall run time of stored procedures.
• Exported data from HDFS to RDBMS via Sqoop for Business Intelligence, visualization and user report generation.
• Developed Nifi flows dealing with various kinds of data formats such as XML, JSON, Avro.
• Worked on importing data from HDFS to MYSQL database and vice-versa using SQOOP.
• Configured Hive meta store with MySQL, which stores the metadata for Hive tables.
• Performed data analytics in Hive and then exported those metrics back to Oracle Database using Sqoop.
• Proactively involved in ongoing maintenance, support and improvements in Hadoop cluster.
• Developed Spark code using Scala and Spark-SQL/Streaming for faster testing and processing of data.
• Used Cloudera Manager for installation and management of Hadoop Cluster.
• Developed data pipeline using Flume, Sqoop, Pig and Java MapReduce to ingest customer behavioral data and financial histories into HDFS for analysis.
• Collaborated with business users/product owners/developers to contribute to the analysis of functional requirements.
• Worked on MongoDB, HBase databases which differ from classic relational databases
• Involved in converting HiveQL into Spark transformations using Spark RDD and through Scala programming.
• Integrated Kafka-Spark streaming for high efficiency throughput and reliability
• Worked on Apache Flume for collecting and aggregating huge amount of log data and stored it on HDFS for doing further analysis.
• Worked in tuning Hive & Pig to improve performance and solved performance issues in both scripts.

Environment: Hadoop 3.0, Hive 2.3, MongoDB, Zookeeper, Spark, MapR, Pig 0.17, Sqoop, Agile, Azure, Jenkins, HDFS, NoSQL, HBase, Impala, MapReduce, YARN, Oozie, Oracle 12c, PL/SQL, Nifi, XML, JSON, MYSQL, Java

Confidential

Big Data/Hadoop Developer

Responsibilities:

• Responsible for building scalable distributed data solutions using Hadoop.
• Used Spark Data Frames Operations to perform required Validations in the data and to perform analytics on the Hive data.
• Developed Apache Spark applications by using spark for data processing from various streaming sources.
• Utilized Agile Scrum Methodology to help manage and organize a team of 4 developers with regular code review sessions.
• Migrated MapReduce jobs to Spark jobs to achieve better performance.
• Extract Real time feed using Kafka and Spark Streaming and convert it to RDD and process data in the form of Data Frame.
• Worked on Kafka and REST API to collect and load the data on Hadoop file system also used Sqoop to load the data from relational databases.
• Wrote Spark-Streaming applications to consume the data from Kafka topics and write the processed streams to HBase.
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs.
• Involved in transforming data from legacy tables to HDFS and Hive tables using Sqoop.
• Expertise in implementing Spark using and Spark SQL for faster testing and processing of data responsible to manage data from different sources Scala.
• Implemented usage of Amazon EMR for processing Big Data across a Hadoop Cluster of virtual servers on Amazon Elastic Compute Cloud (EC2) and Amazon Simple Storage Service (S3).
• Imported data from AWS S3 and into spark RDD and performed transformations and actions on RDD's.
• Implemented Apache Nifi flow topologies to perform cleansing operations before moving data into HDFS.
• Involved in migrating MapReduce jobs into RDD (Resilient data distributions) and create Spark jobs for better performance.
• Developed Spark code using Scala and Spark-SQL/Streaming for faster testing and processing of data.
• Developed the batch scripts to fetch the data from AWS S3 storage and do required transformations in Scala using Spark framework.
• Developed Scala scripts, UDF's using both Data frames/SQL and RDD/MapReduce in Spark for Data Aggregation, queries and writing data back into RDBMS through Sqoop.
• Involved in executing various Oozie workflows and automating parallel Hadoop MapReduce jobs.
• Developed Oozie Bundles to Schedule Pig, Sqoop and Hive jobs to create data pipelines.
• Experience in using ORC, Avro, Parquet, RCFile and JSON file formats and developed UDFs using Hive and Pig.
• Implemented multiple MapReduce Jobs in java for data cleansing and pre-processing.
• Developed Hive queries to do analysis of the data and to generate the end reports to be used by business users.
• Import the data from different sources like HDFS/HBase into Spark RDD and developed a data pipeline using Kafka and Storm to store data into HDFS.
• Used Spark streaming to receive real time data from the Kafka and store the stream data to HDFS using Scala and NoSQL databases such as HBase and Cassandra.
• Documented the requirements including the available code which should be implemented using Spark, Hive, HDFS, HBase and Elastic Search.
• Performed transformations like event joins, filter bot traffic and some pre-aggregations using Pig.
• Explored MLlib algorithms in Spark to understand the possible Machine Learning functionalities that can be used for our use case
• Worked with teams in setting up AWS EC2 instances by using different AWS services like S3, EBS, Elastic Load Balancer, and Auto scaling groups, VPC subnets and CloudWatch.
• Executed Hive queries on Parquet tables stored in Hive to perform data analysis to meet the business requirements.
• Developed code in Java which creates mapping in Elastic Search even before data is indexed into.
• Configured Oozie workflow to run multiple Hive and Pig jobs which run independently with time and data availability.
• Imported and exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
• Created and maintained various Shell and Python scripts for automating various processes and optimized MapReduce code, pig scripts and performance tuning and analysis.

Environment: Hadoop 3.0, Spark, Hive 2.3, Agile, MapReduce, Kafka, HBase, HDFS, Sqoop, Scala, AWS, RDBMS, Oozie, Pig 0.17, Sqoop, Cassandra 3.11, NoSQL, Elastic Search, Java, AWS

Confidential - Union, NJ

Sr. Java/Hadoop Developer

Responsibilities:

• Responsible for requirements gathering, analysis and design and documentation as the application was started from scratch.
• Built data-driven Web applications with server side J2EE technologies like Servlets/JSP and generated dynamic Web pages with Java Server Pages (JSP)
• Involved in implementing the Micro Services based on Restful API utilizing Spring Boot with Spring MVC.
• Worked on MongoDB by using CRUD (Create, Read, Update and Delete), Indexing, Replication and Shading features.
• Implemented Spark RDD transformations to map business analysis and apply actions on top of transformations.
• Developed Spark jobs and Hive Jobs to summarize and transform data.
• Involved in converting Hive/SQL queries into Spark transformations using Spark data frames, Scala and Python.
• Responsible for developing data pipeline with Amazon AWS to extract the data from weblogs and store in Amazon EMR.
• Implemented multiple MapReduce Jobs in java for data cleansing and pre-processing.
• Responsible for developing various modules, front-end and back-end components using several design patterns based on client's business requirements.
• Designed and Developed application modules using spring and Hibernate frameworks.
• Designed and developed the front-end with Swings and Spring MVC framework, Tag libraries and Custom Tag Libraries and development of Presentation Tier.
• Used Hibernate to develop persistent classes following ORM principles.
• Deployed spring configuration files such as application context, application resources and application files.
• Used Java-J2EE patterns like Model View Controller (MVC), Business Delegate, Session façade, Service Locator, Data Transfer Objects, Data Access Objects, Singleton and factory patterns.
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in MapReduce way.
• Developed a custom File System plug in for Hadoop so it can access files on Data Platform.
• Implemented different validation control on the web-pages using AngularJS and developing a cross-platform JavaScript based solution using Angular JS.
• Involved in research, implementation and review of JQuery and JavaScript functions to use plugins for representing data in graphs.
• Developed the persistence layer using Hibernate Framework, created the POJO objects and mapped using Hibernate annotations.
• Developed Maven scripts to build and deploy the application in the WebSphere Application Server.
• Used Spring Data JPA which focuses on JPA to store data in relational database, by which simple Queries are written.
• Involved in deployment of application on WebLogic Application Server.
• Used Jenkins which is an open source continuous integration tool written in java which provides Integration services for software development
• Developed Unit test cases using JUnit and involved in User Acceptance Testing and Bug Fixing.
• Extensively used Eclipse IDE for developing, debugging, integrating and deploying the application.

Environment: scratch, J2EE, Servlets, JSP, spring, Hibernate, MVC, Java, Hive, MapReduce, Hadoop, AngularJS, JavaScript, JQuery, Maven, JUnit, Eclipse, Spark, Scala, Python, AWS

Confidential - Newport Beach, CA

Java/J2EE Developer

Responsibilities:

• Implemented Java/J2EE Design patterns like singleton, MVC, Builder and Strategy patterns.
• Used core java to implement the provided UML diagrams.
• Involved in performance tuning, maintaining coding standards and resolving JVM issues.
• Involved in production support, resolving the production issues and administering the WebSphere server.
• Wrote JUnit test cases for each and every line of the application code and performed validation.
• Configured JNDI resources, Data Base resources, JMS and other configurations on the Application Server.
• Implemented Spring Framework to handle the request from the client side using RESTful web services.
• Integrated Hibernate to access the MySql and MongoDB databases.
• Used Maven as a build tool and retrieved all the dependencies required for the application.
• Involved in developing front-end applications and web interfaces using HTML and JavaScript.
• Used Apache Maven as a build tool for building the RESTful web-service.
• Worked with Jenkins in continuous Integration by using GIT tool and executed projects in Apache Maven and Ant.
• Developed Restful based web services and used JSON as the communication for REST mechanism.
• Handled Java multithreading part in back-end component, one thread will be running for each user, which serves the user.
• Wrote JavaScript code using AJAX, JSON features and used the annotation capabilities of the same for Restful services.
• Used JSTL and custom tag library with frameworks such as Ajax and JQuery, to build interactive and attractive user interfaces.
• Developed various java objects (POJO) as part of persistence classes for ORM.
• Worked on JavaScript, JQuery and Servlets to generate dynamic web pages and web contents.
• Used MVC design pattern for designing application, JSP and CSS as the view component.

Environment: Java, J2EE, MVC, WebSphere, JNDI, spring, RESTful, MySql, MongoDB, Hibernate, Maven, HTML, JavaScript, Jenkins, Ant, JSON, Ajax, JQuery

Confidential

Java/J2EE Developer

Responsibilities:

• Developed and utilized J2EE Services and JMS components for messaging communication in WebSphere Application Server.
• Implemented MVC architecture by separating the business logic from the presentation layer.
• Developed code using Java, J2EE, and spring also used Hibernate as an ORM tool for object relational mapping.
• Used JNDI to perform lookup services for the various components of the system.
• Created REST web services to send data in JSON format to different systems using spring boot.
• Extensively used JQuery to provide dynamic User Interface and for the client side validations.
• Extensively used Eclipse IDE for developing, debugging, integrating and deploying the application.
• Participated in object-oriented design, development and testing of REST APIs using Java.
• Implemented Dependency Injection (IOC) feature of spring framework to inject dependency into objects.
• Developed data access layer by integrating spring and Hibernate.
• Used Hibernate framework for data persistence. Developed Hibernate objects to persist data into the database.
• Responsible for developing Hibernate configuration and mapping files for Persistent layer (Object and Relational Mapping).
• Developed Object Oriented JavaScript code and responsible for client-side validations using JQuery
• Extensively used Spring IOC features with spring framework for bean injection and transaction management.
• Used Spring Framework as middle tier application framework, persistence strategy using spring support for Hibernate for integrating with database.
• Involved in designing the application using MVC pattern
• Created JDBC data source and connection pooling for the Application and hibernate mapping files when needed.
• Consumed Restful Web Services to establish communication between different applications
• Implemented Business Services using the Core java and spring.
• Wrote object-oriented JavaScript for transparent presentation of both client- and server-side validation.

Environment: J2EE, MVC, Java, Hibernate, JSON, JQuery, Eclipse, spring, JavaScript

TECHNICAL SKILLS:

Hadoop/Big Data Technologies: Hadoop 3.0, HDFS, MapReduce, HBase 1.4, Apache Pig, Hive 2.3, Sqoop 1.4, Apache Impala 2.1, Oozie 4.3, Yarn, Apache Flume 1.8, Kafka 1.1, Zookeeper

Cloud Platform: Amazon AWS, EC2, EC3, MS Azure, Azure SQL Database, Azure SQL Data Warehouse, Azure Analysis Services, HDInsight, Azure Data Lake, Data Factory

Hadoop Distributions: Cloudera, Hortonworks, MapR

Programming Language: Java, Scala, Python 3.6, SQL, PL/SQL, Shell Scripting, Storm 1.0, JSP, Servlets

Frameworks: Spring 5.0.5, Hibernate 5.2, Struts 1.3, JSF, EJB, JMS

Web Technologies: HTML, CSS, JavaScript, JQuery 3.3, Bootstrap 4.1, XML, JSON, AJAX

Databases: Oracle 12c/11g, SQL

Operating Systems: Linux, Unix, Windows 10/8/7

IDE and Tools: Eclipse 4.7, NetBeans 8.2, IntelliJ, Maven

NoSQL Databases: HBase 1.4, Cassandra 3.11, MongoDB, Accumulo

Web/Application Server: Apache Tomcat 9.0.7, JBoss, Web Logic, Web Sphere

SDLC Methodologies: Agile, Waterfall

Version Control: GIT, SVN, CVS