SUMMARY

• Over 8 years of expertise in the various product application development, Agile, systems programming, requirements gathering, technical documentation writing, Big Data solutions design and development of a different enterprise application systems, extensive design and implementation exposure practices across various domains
• 4 plus years of experience with full project lifecycle development in J2EE technologies including Software Design, Analysis, Coding, Development, Testing, Implementation, Maintenance and Support
• Sound knowledge of Java, J2EE, & its related technology components
• Around 4 years of experiences in Hadoop Eco - system components HDFS, map-reduce (MRV1, YARN), Pig, Hive, Hbase, Sqoop, Flume, Impala, Oozie, Zookeeper, Strom, Tez, Cassandra, and Programming in spark using Scala
• Well versed and experienced in data architecture including data ingestion pipeline design, Hadoop information architecture
• Solid experience in Hadoop/Big Data related technologies in Storage, Querying, Processing and analysis of data
• Sound knowledge in programming Scala and Spark
• Good knowledge on spark components like Spark SQL, MLib, Spark Streaming and GraphX
• Involved in implementing SparkSQL
• Having experience onspark performance tuning Options
• Hands on experience in data cleaning, transformation using Talend
• Hands on experience in pushing data as delimited files into HDFS using Talend Big data studio
• Involved in Usage of different Talend Hadoop Component like Hive, Pig, Spark
• Load and transform data into HDFS from large set of structured data from /Oracle/SQL server/Teradata using Talend Big data studio
• Design, develop, unit test, and support ETL mappings and scripts for data marts using Talend
• Checking & fixing delimiter in ASCII Files with Talend
• Creating mapping from Source to Target in Talend
• Develop ETL job in Talend to load data from ASCII, Flat files
• Creating parameterize scripts in SQL to apply data Transformation
• Skilled in developing Java Map Reduce programs using Hadoop java API and also using hives and pig to perform data analysis, data cleaning and data transformation.
• In-depth understanding and good hands on experience in Inter Process Communication and message passing using distributed cache
• Developed simple to complex Map/Reduce jobs using Hive and Pig to handle files in multiple formats (JSON, Text, XML, Avro, Sequence File and etc..) worked extensively on creating combiners, Partitioning, Distributed cache to improve the performance of MapReduce jobs
• Experience in working with different data sources like Flat files, XML files, log files and Databases
• Implemented optimized map side join in map reduce to handle different data sets
• Implemented custom writable, comparators and custom input format in MapReduce to handle business specific format
• In Depth understanding in Serialization, Deserialization, Comparator and Comparable Concepts
• Experienced in handling Avro data files in MapReduce programs using Avro data serialization system.
• Hands on experience in dealing with Compression Codecs like Snappy, Gzip.
• Good hands on experience in NoSQL databases such as Hbase and Cassandra
• Implemented Bloom filters in Cassandra using key space creation
• Involved in writing Cassandra CQL statements
• Good hands on experience in developing concurrency using spark and Cassandra together
• Hands on experience in Capturing data from existing rational databases (Oracle, MySQL, SQL and Teradata) that provide SQL interfaces using Sqoop
• Involved in schema designing
• Hands on experience in loading unstructured data into HDFS using Flume/Kafka
• Developed producers for Kafka which compress, and bind many small files into a larger Avro and Sequence files before writing to HDFS to make best use of a Hadoop block size
• Good knowledge in web services
• Good Experience in managing and reviewing Hadoop log files
• Experienced in developing custom UDFs for Pig and Hive using Java by extending core functionality
• Expertise in Real time data ingestion into HBASE and HIVE using Storm
• In-depth understanding of Data Structures and Design Analysis of Algorithms
• Experience with Oozie Workflow Engine in running workflow jobs with actions that run Hadoop Map/Reduce and Pig jobs
• Performed data analytics using PIG and Hive for Data Architects in our team
• Experienced with performing analytics on Time Series data using HBase and Java API.
• Sound knowledge in Elastic Search and Kibana
• Excellent understanding of Object Oriented design methodology and Core Java concepts such as multi-threading, exception handling, generics, annotations, Collections and I/O
• Experience in developing Maven and ANT scripts to automate the compilation, deployment and testing of web application

TECHNICAL SKILLS

Hadoop Ecosystems: CDH3, CDH4, CDH5, HDFS, MapReduce, YARN, Pig, Hive, Hbase, Sqoop, Flume, Impala, Oozie, zookeeper, HUE, Apache Tez, Spark, Scala, Kafka and storm

Data Discovery Applications: Elastic Search and Kibana

NoSQL Databases: Hbase and Cassandra

Hadoop Distributions: CDH3, CDH4, CDH5, Hortonworks and BigInsights

Machine Learning Algorithms: WEKA and Mahout

Build Management Tools: Maven, Apache Ant, GitHub and Version Control SVN

Programming Languages: Java, C, C++, Data Structures, Scala and Python

Java Technologies: JDBC, NetBeans, Servlets, JSP and AJAX

Web User Interfaces: HTML, DHTML, JavaSript, and XML

Web Servers: Apache Tomcat, Web Logic and Web Sphere

Web Services: REST, Jersy, SOAP, WSDL, UDDL

Databases: Oracle 11g, MySQL, IBM DB2, SQL and Teradata

Development Tools: Eclipse, Putty and Tectia

Scripting Languages: Shell Script, Perl, Ant Script and Unix Scripting

Network Protocols: TCP/IP, UDP, HTTP, DHCP, DNS including Socket Programming

PROFESSIONAL EXPERIENCE

Hadoop Developer

Confidential, KY

Responsibilities:

• All the data was loaded from our relational DBs to HIVE using Sqoop. We were getting four flat files from different vendors.
• These were all in different formats e.g. text, EDI and XML formats
• So the vendors were sending those files to one SCE person and SCE person send those files to local file system. From local file system we were loading data to HDFS
• We were getting on average of 110 GB on data on daily basis. On the whole the data warehouse for my project was having 3-4 PB of data and we used 135 nodes culture to process the data
• Since our cluster was over 100 nodes, we were using resource manager (application master)
• We create table in HIVE and after that load data from HDFS to HIVE
• I was writing hive join query to fetch info from multiple tables
• I was writing multiple map-reduce jobs to collect output from hive
• Data (flat files) which we are getting in different format were converted into CSV using Map-reduce framework
• I was also writing UDF for data formatting
• We are also getting duplicate data in files which was corrected by writing map-reduce job
• Involved in developing Map-reduce framework, writing queries scheduling map-reduce
• Large data sets in parallel across the Hadoop cluster for pre-processing
• Written Java projects and used Talend data preperation tool for pre processing of huge data sets
• Developed the code for Importing and exporting data into HDFS and Hive using Sqoop
• Implemented concurrency programming in Spark usig Scala with Message passing and Actor concepts usig scala
• Implemented distributed collection of objects that can be cached in memory across 80 node cluster
• Implemented pre-defined operators in spark such as map, reduce, sample, filter, count, cogroup, groupBy, sort, reduceByKey, take, groupByKey, union, leftOuterJoin, rightOuterJoin, and etc.
• Responsible for writing Hive Queries for analyzing data in Hive warehouse using Hive Query Language (HQL)
• Involved in defining job flows using Oozie for scheduling jobs to manage apache Hadoop jobs by directed
• Directed Acyclic graph (DAG) of actions with control flows
• Involved in loading data into cassandra NoSQL Database
• Good hands on experinece in cassandra project development using keyspace creation
• Worked with Compaction, Bloom filter using cassandra
• Good understanding of SEDA (Staged Event Driven Architecture) architecture
• Implemented record level atomicity on writes using cassandra
• Developed consurrency and cashing using cassandra
• Worked with canssandra web console
• Developed a data pipeline using Kafka and Storm to store data into HDFS
• Developing Hive User Defined Functions (UDFs) in java, compiling them into jars and adding them to the HDFS and executing them with Hive Queries
• Experienced in managing and reviewing Hadoop log files
• Responsible to manage data coming from different sources
• Assisted in monitoring the Hadoop cluster using Ganglia tool ealing with high volume of data in the cluster
• Tested and reported defects in an Agile Methodology perspective
• Consolidate all defects, report it to PM/Leads for prompt fixes by development teams and drive it to closure
• Importing data from Oracle to HDFS & Hive for analytical purpose
• Analyzing imported data in HDFS & Hive using HiveQL and custom Map Reduce programs in Java
• It is team of 65 people including Hadoop developer, Java developer, QA, Data analyst, BA and Project Manager

Environment: Java, CDH5, Hadoop, HDFS, MapReduce, Hive, Spark, Scala, Kafka, Cassandra, Strom, Talend and Java 1.7

Hadoop Developer

Confidential, Detroit, MI

Responsibilities:

• Written multiple java MapReduce jobs for data cleaning and preprocessing
• Experienced in defining job flows using Oozie
• Experienced in managing and reviewing Hadoop log files
• Load and transform large sets of structured, semi structured and unstructured data
• Responsible to manage data coming from different sources and application
• Supported Map Reduce Programs those are running on the cluster
• Involved in loading data from UNIX file system to HDFS.
• Involved in designing schema, writing CQL’s and loading data using Cassandra
• Good experience with CQL Data manipulation commands and CQL clauses
• Worked with CQL collections
• Installed and configured Hive and also written Hive UDFs.
• Involved and experienced with Datastax
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way
• Developed Map Reduce jobs to automate transfer of data from/to HBase
• Assisted with the addition of Hadoop processing to the IT infrastructure
• Used flume to collect the entire web log from the online ad-servers and push into HDFS
• Implemented Map/Reduce job and execute the MapReduce job to process the log data from the ad-servers
• Wrote efficient map reduce code to aggregate the log data from the Ad-server
• Used Hive to analyse the partitioned and bucketed data and compute various metrics for reporting
• Working knowledge in writing PIG's Load and Store functions

Environment: Hortonworks, MapReduce, HDFS, Hive, Pig, Flume, Oozie, Cassandra, Java 1.5, Elastic Search and Kibana

Hadoop Developer

Confidential, Detroit, MI

Responsibilities:

• Gathered the business requirements from the Business Partners and Subject Matter Experts
• Responsible to manage data coming from different sources
• Involved in HDFS maintenance and loading of structured and unstructured data
• Wrote MapReduce job using Java API
• Involved in managing and reviewing Hadoop log files
• Imported data using Sqoop to load data from MySQL to HDFS on regular basis
• Developing Scripts and Batch Job to schedule various Hadoop Program
• Written Hive queries for data analysis to meet the business requirements
• Creating Hive tables and working on them using Hive QL.
• Utilized Agile Scrum Methodology to help manage and organize a team of 4 developers with regular code review sessions
• Weekly meetings with technical collaborators and active participation in code review sessions with senior and junior developers
• Used JUnit for unit testing and Continuum for integration testing

Environment: Hadoop, MapReduce, HDFS, Hive, PIG, MySQL, Java (jdk1.6), and Junit

Sr. Java Developer / Hadoop Developer

Confidential, Chattanooga, TN

Responsibilities:

• Involved in coding ofJSP pages for the presentation of data on the View layer in MVC architecture
• Involved in requirements gathering, analysis and development of the Insurance Portal application
• UsedJ2EE design patternslike Factory Methods, MVC, and Singleton Pattern that made modules and code more organized, flexible and readable for future upgrades
• Worked withJavaScriptto perform client side form validations
• Used Struts tag libraries as well as Struts tile framework
• UsedJDBCto access Database with Oracle thin driver of Type-3 for application optimization and efficiency
• UsedData Access Objectto make application more flexible to future and legacy databases
• Actively involved in tuning SQL queries for better performance
• Wrote generic functions to call Oracle stored procedures, triggers, functions
• Used JUnit for the testing the application in the testing servers
• Providing support for System Integration Testing & User Acceptance Testing
• Used Oracle SQL developer for the writing queries or procedures in SQL.
• Involved in resolving the issues routed through trouble tickets from production floor
• Participated in Technical / Functional Reviews
• Involved in Performance Tuning of the application
• Used Log4J for extensible logging, debugging and error tracing
• Need to discuss with the client and the project manager regarding the new developments and the errors
• Involved in Production Support and Maintenance
• Involved in transferring data from MYSQL to HDFS using Sqoop
• Written map-reduce jobs according to the analytical requirements
• I developed java programs to clean the huge datasets and for pre processing
• Responsible in creating Pig scripts and analyzing from the large datasets
• Involved with different kind of files such as text and xml data
• Involved in developing of UDFs in pig scripts
• Interacted and reported the fetched results to BI department

Environment: JDK, J2EE, UML, Servlet, JSP, JDBC, Struts, XHTML, JavaScript, MVC, XML, XML,Schema, Tomcat, Eclipse, CDH, Hadoop, HDFS, Pig, MYSQL and MapReduce

Java Developer

Confidential, Pittsburgh, PA

Responsibilities:

• Involved in requirement gathering & Analysis of the project
• Designed the functional specifications and architecture of the web-based module using Java Technologies
• Created Design specification using UML Class Diagrams, Sequence & Activity Diagrams
• Developed the Web Application using MVC Architecture, Java, JSP, and Servlets & Oracle Database
• Developed various Java classes, SQL queries and procedures to retrieve and manipulate the data from backend Oracle database using JDBC.
• Extensively worked with Java Script for front-end validations
• Analysis of business requirements and develop system architecture document for the enhancement project
• Designed and developed applications on Service Oriented Architecture (SOA)
• Created UML (Use cases, Class diagrams, Activity diagrams, Component diagrams, etc.) using Visio
• Provided Impact Analysis and Test cases
• Delivered the code within the timeline, and logged the bugs/fixes in TechOnline, tracking system
• I had developed Unit & Functional Test cases for testing Web Application
• Used Spring (MVC) architecture to implement the application using the concrete principles laid down by several design patterns such as Composite View, Session Facade, Business Delegate, Bean Factory, Singleton, Data Access Object and Service Locator
• Involved in the integration of spring for implementing Dependency Injection
• Developed code for obtaining bean references in Spring IOC framework
• Focused primarily on the MVC components such as Dispatcher Servlets, Controllers, Model and View Objects, View Resolver
• Involved in creating the Hibernate POJO Objects and utilizing Hibernate Annotations
• Used Hibernate, object/relational-mapping (ORM) solution, technique of mapping data

Environment: Windows NT 2000/2003, XP, and Windows 7/ 8, Java, UNIX, SQL, SOA, JDBC, JavaScript, Maven, JUnit, Agile/Scrum Methodology and SVN Version Control

JAVA Developer

Confidential

Responsibilities:

• Developed web components using JSP, Servlets and JDBC
• Designed tables and indexes
• Designed, Implemented, Tested and Deployed Enterprise Java Beans both Session and Entity using WebLogic as Application Server
• Developed stored procedures, packages and database triggers to enforce data integrity. Performed data analysis and created crystal reports for user requirements
• Implemented the presentation layer with HTML, XHTML and JavaScript
• Used EJBs to develop business logic and coded reusable components in Java Beans
• Development of database interaction code to JDBC API making extensive use of SQL
• Query Statements and advanced Prepared Statements
• Used connection pooling for best optimization using JDBC interface
• Used EJB entity and session beans to implement business logic and session handling and transactions Developed user-interface using JSP, Servlets, and JavaScript
• Wrote complex SQL queries and stored procedures
• Actively involved in the system testing
• Prepared the Installation, Customer guide and Configuration document which were delivered to the customer along with the product
• Responsible for creating work model using HTML and JavaScript to understand the flow of the web application and created class diagrams.
• Participated in the daily stand up SCRUM agile meetings as part of AGILE process for reporting the day to day developments of the work done
• Design and develop user interfaces using HTML, JSP.
• J2EE is used to develop the application based on MVC architecture
• Created interactive front-end GUI using JavaScript, JQuery, DHTML and Ajax
• Used SAX and DOM XML parsers for data retrieval

Environment: Windows NT 2000/2003, XP, and Windows 7/ 8 C, Java, JSP, Servlets, JDBC, EJB, DOM, XML, SAX

Java Developer

Confidential

Responsibilities:

• Successfully completed the Architecture, Detailed Design & Development of modules Interacted with end users to gather, analyze, and implement the project
• Developed applications that enable the public to review the Inventory Management
• Established schedule and resource requirements by planning, analyzing and documenting development effort to include time lines, risks, test requirements and performance targets
• Analyzing System Requirements and preparing System Design document
• Developing dynamic User Interface with DHTML and JavaScript using JSP and Servlet Technology
• Designed and developed a sub system where Java Messaging Service (JMS) applications are developed to communicate with MQ in data exchange between different systems
• Java Message Oriented Middleware (MOM) API for sending messages between clients
• Used JMS elements for sending and receiving messages
• Used hibernate for mapping from Java classes to database tables
• Wrote PL/SQL & SQL in Oracle Database for creating tables, indexes, triggers and query statements
• Design and develop enterprise web applications, for internal production support group, using Java (J2EE), Design Patterns and Struts framework
• Tuning and Index creation for improved performance
• Designed and developed database schema for new applications
• Created connection pooling method to avoid the waiting for database connection
• Designed an ER Diagram for all the databases using the DB Designer an Open Source Tool
• Designed the Class Diagrams and the Use cases Diagram using the Open Source tool
• Created and executed Test Plans using Quality Center by Test Director
• Mapped requirements with the Test cases in the Quality Center
• Supporting System Test and User acceptance test
• Created internal users and support personnel by drafting and reviewing company product documentation such as Technical document, Impact assessment document

Environment: Rational Application Developer 6.0, Rational Rose, Java, J2ee, JDBC, EJB, JSP, EL, JSTL, JUNIT, XML, SOAP, WSDL, SOA.