SUMMARY:

• Overall 9+ years of experience as a Sr. Big Data Developer with skills in analysis, design, development, testing and deploying various software applications.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice - versa.
• Good knowledge in using Hibernate for mapping Java classes with database and using Hibernate Query Language (HQL).
• Hands on experience in configuring and working with Flume to load the data from multiple sources directly into HDFS.
• Good experience in developing MapReduce jobs in J2EE /Java for data cleansing, transformations, pre-processing and analysis.
• Good Knowledge in Amazon Web Service (AWS) concepts like EMR and EC2 Webservices which provides fast and efficient processing of Teradata Big Data Analytics.
• Experience in collection of Log Data and JSON data into HDFS using Flume and processed the data using Hive/Pig.
• Extensive experience on developing Spark Streaming jobs by developing RDD's (Resilient Distributed Datasets) and used SparkSQL as required.
• Experience on developing Java MapReduce jobs for data cleaning and data manipulation as required for the business.
• Strong knowledge on Hadoop eco systems including HDFS, Hive, Oozie, HBase, Pig, Sqoop, Zookeeper etc.
• Extensive experience with advanced J2EE Frameworks such as spring, Struts, JSF and Hibernate.
• Expertise in JavaScript, JavaScript MVC patterns, object Oriented JavaScript Design Patterns and AJAX calls.
• Installation, configuration and administration experience in Big Data platforms Cloudera Manager of Cloudera, MCS of MapR.
• Experience working with Horton works and Cloudera environments.
• Good knowledge in implementing various data processing techniques using Apache HBase for handling the data and formatting it as required.
• Excellent experience in installing and running various Oozie workflows and automating parallel job executions.
• Experience on Spark and SparkSQL, Spark Streaming, Spark GraphX, Spark MLlib.
• Extensively development experience in different IDE like Eclipse, NetBeans, IntelliJ.
• Strong experience in core SQL and Restful web services (RWS).
• Strong knowledge in NOSQL column oriented databases like HBase and its integration with Hadoop cluster.
• Good experience in Tableau for Data Visualization and analysis on large datasets, drawing various conclusions.
• Experience in using Python, R for statistical analysis.
• Good knowledge of coding using SQL, SQL Plus, T-SQL, PL/SQL, Stored Procedures/Functions.
• Worked on Bootstrap, AngularJS and NodeJS, knockout, ember, Java Persistence Architecture (JPA).
• Well versed working with Relational Database Management Systems as Oracle, MSSQL, MySQL Server.
• Experience with all stages of the SDLC and Agile Development model right from the requirement gathering to Deployment and production support.
• Experience in using PL/SQL to write Stored Procedures, Functions and Triggers.

TECHNICAL SKILLS:

Hadoop/Big Data Technologies: Hadoop 3.0, HDFS, MapReduce, HBase 1.4, Apache Pig 0.17, Hive 2.3, Sqoop 1.4, Apache Impala 3.0, Oozie 4.3, Yarn, Apache Flume 1.8, Kafka 1.1, Zookeeper 3.4

Hadoop Distributions: Cloudera, Hortonworks, MapR

Cloud: AWS, Azure, Azure SQL Database, Azure SQL Data Warehouse, Azure Analysis Services, HDInsight, Azure Data Lake and Data Factory.

Programming Language: Java, Scala 2.12, Python 3.6, SQL, PL/SQL, Shell Scripting, Storm 1.0, JSP, Servlets

Frameworks: Spring 5.0.5, Hibernate 5.2, Struts 1.3, JSF, EJB, JMS

Web Technologies: HTML5, CSS, JavaScript, JQuery 3.3, Bootstrap 4.1, XML, JSON, AJAX

Databases: Oracle 12c/11g, SQL, Neo4j

Database Tools: TOAD, SQL PLUS, SQL

Operating Systems: Linux, Unix, Windows 10/8/7

IDE and Tools: Eclipse 4.7, NetBeans 8.2, IntelliJ, Maven

NoSQL Databases: HBase 1.4, Cassandra 3.11, MongoDB

Web/Application Server: Apache Tomcat 9.0.7, JBoss, Web Logic, Web Sphere

SDLC Methodologies: Agile, Waterfall

Version Control: GIT, SVN, CVS

WORK EXPERIENCE:

Confidential - Merrimack, NH

Sr. Big Data Developer

Responsibilities:

• As a Sr. Big Data Developer, I worked on Hadoop eco-systems including Hive, HBase, Oozie, Yarn, Spark Streaming MCS (MapR Control System) and so on with MapR distribution.
• Installed and configured Hadoop MapReduce, HDFS, Developed multiple MapReduce jobs in Java for data cleaning and preprocessing.
• Primarily involved in Data Migration process using Azure by integrating with Bitbucket repository and Jenkins.
• Built code for real time data ingestion using Java, MapR-Streams (Kafka) and STORM.
• Involved in various phases of development analyzed and developed the system going through Agile Scrum methodology.
• Worked on Apache Solr which is used as indexing and search engine.
• Involved in development of Hadoop System and improving multi-node Hadoop Cluster performance.
• Worked on analyzing Hadoop stack and different Big Data tools including Pig and Hive, HBase database and Sqoop.
• Developed data pipeline using flume, Sqoop and pig to extract the data from weblogs and store in HDFS
• Worked with different data sources like Avro data files, XML files, JSON files, SQL server and Oracle to load data into Hive tables.
• Implemented usage of Amazon EMR for processing Big Data across Hadoop Cluster of virtual servers on Amazon Elastic Compute Cloud (EC2) and Amazon Simple Storage Service (S3)
• Performed transformations, cleaning and filtering on imported data using Hive, MapReduce, Impala and loaded final data into HDFS.
• Developed Python scripts to find vulnerabilities with SQL Queries by doing SQL injection.
• Experienced in designing and developing POC's in Spark using Scala to compare the performance of Spark with Hive and SQL/Oracle.
• Responsible for coding MapReduce program, Hive queries, testing and debugging the MapReduce programs.
• Extracted Real time feed using Spark streaming and convert it to RDD and process data into Data Frame and load the data into Cassandra.
• Involved in the process of data acquisition, data pre-processing and data exploration of telecommunication project in Scala.
• Implemented a distributed messaging queue to integrate with Cassandra using Apache Kafka and Zookeeper.
• Specified the cluster size, allocating Resource pool, Distribution of Hadoop by writing the specification texts in JSON File format.
• Imported weblogs & unstructured data using the Apache Flume and stores the data in Flume channel.
• Exported event weblogs to HDFS by creating a HDFS sink which directly deposits the weblogs in HDFS.
• Used RESTful web services with MVC for parsing and processing XML data.
• Utilized XML and XSL Transformation for dynamic web-content and database connectivity.
• Involved in loading data from UNIX file system to HDFS. Involved in designing schema, writing CQL's and loading data using Cassandra.
• Built the automated build and deployment framework using Jenkins, Maven etc.

Environment: Hadoop 3.0, Hive 2.3, HBase 1.2, Oozie 4.3, MapR, MapReduce, HDFS, Azure, Kafka 2.1, Java, JSON, Spark 2.4, Scala 2.12, Cassandra 3.11, Zookeeper 3.4, MVC, Jenkins, Maven

Confidential - Lowell, AR

Big Data/Hadoop Developer

Responsibilities:

• Responsible for building scalable distributed data solutions using Hadoop.
• Worked as Big Data/Hadoop Developer and admin in Hortonworks distribution for 10 clusters ranges from POC to PROD.
• Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage & review log files
• Used Spark API over Cloudera Hadoop YARN to perform analytics on data in Hive.
• Involved in Agile methodologies, daily scrum meetings, spring planning.
• Integrate visualizations into a Spark application using Data Bricks and popular visualization libraries.
• Developed data pipeline using Flume, Sqoop to ingest customer behavioral data and purchase histories into HDFS for analysis.
• Continuous monitoring and managing the Hadoop cluster using Cloudera Manager.
• Used HiveQL for data analysis like creating tables and import the structured data to specified tables for reporting.
• Used Pig to perform data validation on the data ingested using scoop and flume and the cleansed data set is pushed into HBase.
• Participated in development/implementation of Cloudera Hadoop environment.
• Developed Spark code using Scala and Spark-SQL/Streaming for faster processing of data.
• Prepared Spark builds from MapReduce source code for better performance.
• Imported data using Sqoop to load data from MySQL to HDFS on regular basis.
• Developed Scripts and Batch Job to schedule various Hadoop Program.
• Used Spark API over Hortonworks, Hadoop YARN to perform analytics on data in Hive.
• Wrote Hive queries for data analysis to meet the business requirements.
• Developed Kafka producer and consumers for message handling.
• Executed Hadoop/Spark jobs on AWS EMR using programs, data stored in S3 Buckets.
• Exploring with Spark improving performance and optimization of the existing algorithms in Hadoop MapReduce using Spark Context, Spark-SQL, Data Frames, Pair RDD's and Spark YARN.
• Deployed MapReduce and Spark jobs on Amazon Elastic MapReduce using datasets stored on S3.
• Used Amazon Cloud watch to monitor and track resources on AWS.
• Responsible for developing data pipeline using Flume, Sqoop and Pig to extract the data from weblogs and store in HDFS.
• Experienced with batch processing of data sources using Apache Spark and Elastic search.
• Experienced in implementing Spark RDD transformations, actions to implement business analysis.
• Implemented Spark using Scala and Spark SQL for faster testing and processing of data.
• Loaded data into the cluster from dynamically generated files using Flume and from relational database management systems using Sqoop.
• Involved in importing the real-time data to Hadoop using Kafka and implemented Oozie jobs for daily imports.
• Automated the process for extraction of data from warehouses and weblogs by developing work-flows and coordinator jobs in Oozie.
• Created Hive tables and involved in data loading and writing Hive UDFs.

Environment: Hadoop 3.0, Spark 2.4, Hive 2.3, Agile, Apache Flume 1.8, HDFS, HiveQL, Pig 0.17, MapReduce, MySQL 2016, Hortonworks, AWS, Kafka 2.1, Oozie 4.3

Confidential - Bellevue, WA

Hadoop Developer

Responsibilities:

• Wrote JavaScript code using AJAX, JSON features and also used the annotation capabilities of the same for Restful services.
• Used JSTL and custom tag library with frameworks such as Ajax and jQuery, to build interactive and attractive user interfaces.
• Designed and implemented core Java functionality, including the basic database access layer, generic form handling routines and JSP.
• Designed and guided the team for implementing Client module and XSL conversions using XSLT.
• Installed and configured Hadoop MapReduce, HDFS, Developed multiple MapReduce jobs in java for data cleaning and pre-processing.
• Supported MapReduce Programs those are running on the cluster.
• Involved in loading data from UNIX file system to HDFS.
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in MapReduce way.
• Develop MapReduce jobs for the users. Maintain, update and schedule the periodic jobs which range from updates on periodic MapReduce jobs.
• Build scripts using ANT and MAVEN build tools in Jenkins to move from one environment to other environments.
• Developed applications using Spring Framework for Dependency Injection (Inversion of Control) and implemented the business logic using EJB (session and entity beans).
• Used Declarative Transaction management system of Spring Framework and integrated it with JDBC and Hibernate transaction managers to customize transactional behavior.
• Implemented navigation using Spring MVC controllers, configured controllers using Spring MVC annotations and configuration files.
• Developed presentation layer components comprising of JSP, Servlets and JavaBeans using the Spring framework.
• Connected to the Foreign JMS Server to send the message to the request queue and get the response from the reply queue.
• Designed and Developed XSL style sheets using XSLT to transform XML and display the information on the UI screens.
• Implementing Restful web services architecture for Client-server interaction and implemented respective POJOs for its implementations.
• Developed screens using JSP, DHTML, CSS, Ajax, JavaScript, spring, Java and XML.
• Developed and implemented Swing, spring and J2EE based MVC (Model-View-Controller) framework for the application
• Worked with NoSQL databases like HBase to create tables and store the data Collected and aggregated large amounts of log data using Apache Flume and staged data in HDFS for further analysis.
• Collaborated with Business users for requirement gathering for building Tableau reports per business needs.
• Worked on deploying Hadoop cluster with multiple nodes and different big data analytic tools including Pig, HBase database and Sqoop HDP.
• Used CVS version controlling tool and project build tool using ANT.
• Used Java Message Service (JMS) for reliable and asynchronous exchange of important information, such as order status report, delivery report.
• Developed different Action classes for Business logic implementation which basically calls different Server-side components like EJB, DAO and other Business components.
• Used Hibernate for Object Relational Mapping (ORM) purposes for transparent persistence onto the Oracle database using Hibernate Query language (HQL).
• Designed and developed the Web User Interface content using JSP, JSP custom tags, HTML and used Ajax and JavaScript for client-side validation.

Environment: JavaScript, AJAX, JSON, jQuery, Java 6, Hadoop 2.6, MapReduce, HDFS, Maven, ANT, Jenkins

Confidential - Dallas, TX

Java/J2ee Developer

Responsibilities:

• Responsible to write the different service classes and utility API, which will be used across the framework.
• Involved in developing the application using Java/J2EE platform, implemented the Model View Control (MVC) structure using spring.
• Used Ext.JS framework for the development of web application using AngularJS, Ajax, DHTML and DOM scripting.
• Responsible to enhance the Portal UI using HTML, JavaScript, XML, JSP, CSS as per the requirements and providing the client side using JQuery validations.
• Used React JS to create views to hook up models to the DOM and synchronize data with server as a Single Page Application (SPA).
• Worked with e-commerce databases to develop shopping cart manager.
• Used Spring Core Annotations for Dependency Injection and used Apache Camel to integrate Spring framework.
• Connected Java applications to Java database to read, write data.
• Developed static and dynamic Web Pages using JSP, HTML and CSS.
• Worked on JavaScript for data validation on client side.
• Used Hibernate as Persistence framework mapping the ORM objects to table using Hibernate annotations Used SOAP UI for testing the Web services by sending an SOAP request.
• Used AJAX framework for server communication and seamless user experience.
• Created UNIX shell scripts to automate the build process, to perform regular jobs like file transfers between different hosts.
• Used JUnit for the unit testing of various modules.
• Developed Web services component using XML, WSDL, and SOAP with DOM parser to transfer and transform data between applications.
• Created user-friendly GUI interface and Web pages using HTML, CSS and JSP.
• Developed web components using MVC pattern under Struts framework.
• Wrote JSPs, Servlets and deployed them on Weblogic Application server.
• Wrote the Hibernate-mapping XML files to define java classes-database tables mapping.
• Implemented MVC architecture by using spring to send and receive the data from front-end to business layer.
• Extensively used Hibernate in data access layer to access and update information in the database.
• Migrated the Servlets to the Spring Controllers and developed Spring Interceptors, worked on JSPs, JSTL, and JSP Custom Tags.
• Developed and flexible, scalable, utilizing open source technologies like Hibernate ORM and Spring Framework.
• Responsible for design and implementation of various modules of the application using Spring-Hibernate architecture.

Environment: Java 4, J2EE, MVC, AngularJS, Ajax, HTML, JavaScript, XML, Hibernate

Confidential

Java Developer

Responsibilities:

• Developed customer facing software for managing advertisements to grow the business using JavaScript/JQuery, HTML, and CSS.
• Worked with JMS as messaging service and JTA for Transaction management and having knowledge with JPA for Java persistence.
• Implemented backend process for sending the data through FTP.
• Used spring framework for dependency injection and integrated with Node JS, Hibernate and JSF.
• Developed MVC controllers and deployed flow of Execution in the Servlet Environment using Spring MVC.
• Developed Test Scripts to implement Test Cases, Test Scenarios, and features for BDD (Behavior Driven Development).
• Extensively used Core Spring Framework for Dependency Injection of components.
• Created Selenium web-driver automation scripts using Java to validate functionality on various browsers.
• Implemented Web Services using Spring RESTful.
• Developed Hibernate with Spring Integration as the data abstraction to interact with the database.
• Used Hibernate for mapping java classes with database and created queries with Hibernate query language (HQL).
• Responsible for gathering the requirements, designing and developing the applications.
• Worked on UML diagrams for the project use case.
• Worked with Java String manipulations, to parse CSV data for applications.
• Develop quality code adhering to Scala coding Standards and best practices.
• Worked extensively on AngularJS/NodeJS creating Controllers.
• Developed AJAX applications in java and compiled the source to JavaScript running on all the browsers using GWT.
• Involved in environment setting, which includes installation, configuration application server.
• Created build scripts for running components individually using Apache ANT and XML.

Environment: Apache ANT, XML, JavaScript, CSV, AngularJS, AJAX, Hibernate, MVC, JQuery, HTML, CSS