SUMMARY

• 9 years of IT professional experience with emphasis on Design, Development, Implementation, Testing, Maintenance and Deployment of Software Applications using Java, J2EE and Big Data technologies.
• 4+ years of experience in Big Data technologies and Hadoop ecosystem components like Spark, HDFS, MapReduce, Pig, Hive, YARN, Sqoop, Flume, Kafka and NoSQL systems like HBase, Cassandra.
• Expertise in architecting real time streaming applications and batch style large scale distributed computing applications using tools like Spark Streaming, Kafka, Hive, Impala etc.
• Experience in creating complex data ingestion pipelines, data transformations, data management and data governance at Enterprise level.
• Hands - on experience in installing, configuring and monitoring HDFS clusters (on premise & cloud AWS).
• In depth understanding of Map Reduce and AWS cloud concepts and its critical role in data analysis of huge and complex datasets.
• Created custom Database Encryption Connectors that could be plugged into Sqoop to be able to encrypt the data while importing to HDFS/Hive.
• Experience in fine-tuning and troubleshooting Spark Applications, Hive queries.
• Extensive hands on experience in writing complex MapReduce jobs, Pig Scripts and HiveQL scripts.
• Experience using various Hadoop Distributions (Cloudera, Hortonworks, Amazon AWS) to fully implement and leverage new Hadoop features.
• Experience in using Amazon Cloud services like S3, EMR etc.
• Proficient in using Cloudera Manager, an end-to-end tool to manage Hadoop operations in Cloudera Cluster.
• Experience in Apache Flume and Kafka for collection, aggregation and moving huge chunks of data from various sources such as web server, telnet sources etc.
• Worked on Java HBase API for ingesting processed data to HBase tables.
• Extensive knowledge and experience in using Apache Storm, Spark Streaming, Apache Spark, Kafka and Flume in creating data streaming solutions.
• Experienced in working with Machine learning libraries (spark-MLlib) and implementing ML algorithms for clustering, regression filtering and dimensional reduction.
• Extensive understanding of Partitions and Bucketing concepts in Hive.
• Designed and managed External tables with right partition strategies to optimize performance in Hive.
• Extensive experiences in working with semi/unstructured data by implementing complex MapReduce programs using Combiners, Partitions and design patterns.
• Created few Hive UDF's to perform some complex business specific transformations and rules.
• Expert knowledge over J2EE Design Patterns like MVC Architecture, Session Facade, Front Controller and Data Access Objects for building J2EE Applications.
• Experienced in using Agile methodologies including extreme programming, Scrum Process and Test-Driven Development (TDD).
• Intensive work experience in developing enterprise solutions using Java, J2EE, Struts, Servlets, Hibernate, JavaBeans, JDBC, JSP, JSF, JSTL, MVC, Spring, Custom Tag Libraries, JNDI, AJAX, SQL, JavaScript, AngularJS and XML.
• Conversant with web application Servers like Tomcat, WebSphere, WebLogic and Jboss servers.
• Experience in development of logging standards and mechanism based on Log4j.
• Experience in writing ANT and Maven scripts to build and deploy Java applications.

TECHNICAL SKILLS

Big Data Ecosystem: Hadoop, HDFS, Spark, Hive, Impala, Kafka, Hue, MapReduce, YARN, Pig, Sqoop, HBase, Couchbase, Cassandra, Oozie, Storm, Flume, Talend, AWS, Hortonworks and Cloudera clusters

Programming Languages: Scala, Java, C, UNIX Shell Scripting, AngularJS, PL/SQL, Python

Java Tools & Web Technologies: J2EE, JSF, EJB, HTML, XHTML, AngularJS, Servlets, JSP, CSS, XML, Ajax, Java script, SOAP, RESTful

Database: Couchbase, Cassandra, HBase, Oracle 10g, MySQL, Teradata SQL

Frameworks: MVC, Spring, Struts, JDBC

Visualization: Tableau, MS Excel

Development Tools: IntelliJ, Eclipse, NetBeans, ANT, Maven and SBT

PROFESSIONAL EXPERIENCE

Confidential, Chicago, IL

Sr. Hadoop Developer

Responsibilities:

• Worked on importing and exporting data from Teradata, MySQL into HIVE using Sqoop for visualization, analysis and to generate reports.
• Loaded CSV files containing user event information into Hive External tables on daily basis.
• Created Spark applications and used Data frames and Spark-SQL API primarily for performing event enrichment and performing lookups with other enterprise data sources.
• Build a real time streaming pipeline by using Kafka integration with Storm and Spark Streaming.
• Perform ETL on different formats of data like JSON, CSV files and converted them to parquet while loading to final tables. Ran ad-hoc querying using Hive and Impala.
• Extensively perform complex data transformations in Spark using Scala language.
• Worked extensively with importing metadata into Hive using Scala and migrated existing tables and applications to work on Hive and AWS cloud.
• Involved in converting Hive/SQL queries into Spark transformations using Scala.
• Connect Tableau and Squirrel SQL clients to Spark-SQL (Spark thrift server) via data source and run the queries.
• Worked with Machine learning libraries (Spark MLlib) and done clustering, regression, filtering and dimensional reduction using implemented ML algorithms.
• Configured Spark streaming to receive real time data from the Kafka and store the stream data to HDFS using Scala (Prototype).
• Used Impala as the primary analytical tool for allowing visualization servers to connect and perform reporting on top Hadoop directly.
• Used Machine learning libraries as Spark MLlib and implemented ML algorithms for clustering, regression filtering and dimensional reduction process with data scientists.
• Wrote spark jobs for Data clustering and data processing using Spark-MLlib and cluster algorithms as per functional requirements.
• Experience in large-scale streaming data analytics using Storm.
• Working on Oozie workflow engine to run multiple Hive-QL jobs and on schedulers.
• Done various compressions and file formats like Parquet, Snappy, Gzip, Bzip2, Avro, Text.
• Implemented test scripts to support test driven development and continuous integration.
• Used Zookeeper to provide coordination services to the cluster.
• Used Impala and Tableau to create various reporting dashboards.

Environment: HDFS, Hadoop 2.x, Pig, Hive, Sqoop, Flume, Spark, MapReduce, Scala, Oozie, YARN, Tableau, Squirrel SQL, Spark-SQL, Spark-MLlib, Impala, Cassandra, UNIX Shell Scripting, Storm, Zookeeper, Kafka, Agile Methodology, Cloudera 5.9, SBT.

Confidential, Denver, CO

Sr. Hadoop Developer

Responsibilities:

• Responsible for building scalable distributed data solutions using Hadoop.
• Implemented Installation and configuration of multi-node cluster on Cloud using Amazon Web Services(AWS) on EC2.
• Worked on importing and exporting data from DB2 into Hive using Sqoop.
• Involved in importing the real-time data toHadoopusing Kafka and implemented the Oozie job for daily imports.
• Developed Hive scripts to perform data transformation and etl processes.
• Developed MapReduce (YARN) programs to cleanse the data in HDFS obtained from assorted data sources to make it suitable for ingestion into Hive schema for analysis.
• Designed and developed MapReduce jobs to process data coming in different file formats like XML, CSV, JSON.
• Transferring data between MySQL and Hadoop Distributed File System (HDFS) using Sqoop with connectors.
• Creating and populating Hive tables and writing Hive queries for data analysis to meet the business requirements.
• Running MapReduce jobs to access HBase data from application using Java Client APIs.
• Developed Pig Latin scripts and HQL queries for the analysis of Structured, Semi-Structured and Unstructured data.
• Automating the extraction, processing and analysis of data jobs using Oozie.
• Used SVN for version control.
• Created working POC’s usingSpark 1.1.0 streamingfor real time stream processing of continuous stream of large data sets.
• Involved building and managing NoSQL Database like Hbase or Cassandra.
• Integrating Cassandra with Storm for real time user attributes look up.
• Worked in Spark to read the data from Hive and write it to Cassandra using Java.
• Developed custom aggregate functions using Spark SQL and performed interactive querying.
• Developed Spark scripts by using Scala and Python Shell commands as per the requirement.
• Installed and configured Hadoop Scala, HDFS (AWS cloud) and responsible for Developing multiple Spark jobs in Scala, R scripts & CLI for data managing and preprocessing events.
• Implemented Machine learning libraries, spark-MLlib algorithms for clustering, filtering, regression and dimensional reduction.
• Worked on data warehouse product Amazon Redshift which is a part of the AWS (Amazon Web Services)
• Developed Spark and SparkSQL scripts to migrate data from RDBMS into AWS-RedShift.
• Developed ETL Scripts for Data acquisition and Transformation using Talend.

Environment: Hadoop 2.x, Hive, HQL, HDFS, MapReduce, Spark 1.1.0, Scala, Sqoop, Storm, Kafka, Flume, Oozie, HBase, AWS-RedShift, Python, Java, Maven, Eclipse, Putty, AWS-EC2, Talend, Hortonworks.

Confidential, Minnetonka, MN

Hadoop Developer

Responsibilities:

• Analyzing the functional specifications provided by the client and developing detailed solution design document with the Architect and the team.
• Used Hadoop architecture with Map Reduce functionality and its ecosystem to solve the customer requirements using Cloudera Distribution for Hadoop (CDH).
• Developed multiple Map Reduce jobs in Java for complex business requirements including data cleansing and preprocessing.
• Created Data Lake which serves as a base layer to store and do analytics on data flowing from multiple sources into Hadoop Platform.
• Involved in creating Hive tables, loading the data and writing hive queries which will run internally in map reduce.
• Optimized Hive tables using optimization techniques like partitions and bucketing to provide better performance with HiveQL queries.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Scripted complex HiveQL queries on Hive tables for analytical functions by implementing Hive Generic UDFs.
• Wrote customized User Defined Function’s (UDF) to process further in Java and Perl to ease the processing in Pig.
• Involved in building applications using Maven and integrated them by using continuous Integration servers like Jenkins to build jobs.
• Worked on Import and export data from Legacy Databases RDBMS into HDFS and Hive using Sqoop.
• Worked extensively with importing metadata into Hive and migrated existing tables and applications to work on Hive and AWS cloud.
• Designing NoSQL schemas in HBase and Cassandra.
• Involved in Agile methodologies, daily Scrum meetings, Sprint planning.

Environment: Hadoop 1.x, HDFS, MapReduce, Hive, Pig, HBase, Tez, Sqoop, Oozie, Maven, Shell Scripting, Teradata, CDH3, Cloudera Manager.

Confidential, Raleigh, NC

Java Application Developer

Responsibilities:

• Involved in distinct phases of Software Development Life Cycle as requirement gathering, data modeling, analysis, architecture design & development for the project.
• Worked in Agile Environment and responsibilities include analysis of these various applications, co-ordination with client, meetings with business users.
• Used JSF Framework and Spring Framework in the application, which is based on MVC design pattern.
• Spring Framework is used for Dependency injection, security and integration with the Hibernate framework.
• Written the Java Script, HTML, DHTML, CSS, Servlets, and JSP for designing GUI of the application.
• Designed and developed user interface using JSP, JSTL, JSF and Custom Tag Libraries and AJAX JQuery, Angular JS to speed the application.
• Extensively used Hibernate in the database to access and update information in data access layer.
• Used Angular JS, JQuery to manipulate the DOM objects for Ajax calls and for User interface look and feel.
• Extensive knowledge of database practices, structures, principles and theories.
• Used PL/SQL to manage data and create tables and performed unit testing using Junit framework
• Implemented RESTFUL web services using Angular JS.
• Used MAVEN to build the application.
• Monitored the error logs using Log4J and fixed the problems.
• Developed the different components of application using STS 3.4 and used SVN for version control.

Environment: Java/J2EE, Tomcat 7.0, Spring 3.1, Hibernate 3.2, JavaScript, HTML, DHTML, CSS, Servlets, JSP, JSTL, XML, SOAP, Web Services, PL/SQL, JDBC, JQuery, Ajax, Maven, MVC, Unix, Angular JS, Log4J.

Confidential

Java/ J2EE Developer

Responsibilities:

• Developed the application using Agile methodology and Scrum method of project management.
• Involved in group meetings and made substantial changes to the design to improve performance of the Application.
• Responsible for front-end UI design using HTML/HTML5, CSS/CSS3, JavaScript, jQuery, etc., taking advantage of the MVC pattern of the Spring MVC to produce higher maintainability of the code.
• Developed UI screens using JSP, JQuery, JavaScript, XHTML, CSS.
• Used JSON encoding and decoding techniques to convert complex objects to string and vice versa.
• Web service methods were written using REST API.
• Developed and deployed Enterprise Web Services (SOAP and RESTFUL) and generated client using Jersey and Axis Frameworks using Eclipse.
• Extensively used Core Java concepts like Multithreading, Collections Framework, File I/o and concurrency.
• Business layer was integrated using Hibernate and Oracle 10g.
• Worked on design patterns like delegate, service layer and various internal design frameworks -links, notification and audit frameworks.
• Developed and executed unit test cases using JUnit, and Mockito as mocking framework for mocking data.
• Used GIT for version control and JIRA issue tracker to file the bugs.
• Used MAVEN for building the application and deployed on Tomcat Server.
• Actively involved in code reviews and in bug fixing.

Environment: Core Java, J2EE 1.6, Spring Framework, Bootstrap, HTML5, Java script, JQuery, CSS, Mockito, Apache Tomcat 7, Eclipse, XML, Maven, Log4j, REST API, Hibernate, Oracle, Junit, GIT, JIRA, UML and Apache AXIS.