SUMMARY

• Over all 8+ years of experience in IT which includes 3+ years' experience using Apache Hadoop and experience for analyzing the Big Data as per the requirement and 2+years’ experience in Data warehouse implementation and 3+ years’ experience as a Java developer.
• Good Exposure on Apache Hadoop Map Reduce programming, Hive, PIG scripting and HDFS.
• Hands on experience in installing, configuring, monitoring, and using Hadoop ecosystem components like Hadoop MapReduce, HDFS, HBase, Hive, Sqoop, Pig, Zookeeper, Horton works, Flume, spark.
• In depth knowledge of understanding the Hadoop architecture and its components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, Resource Manager, Node Manager, Map Reduce programs and YARN paradigm.
• Worked on developing ETL processes to load data from multiple data sources to HDFS using FLUMEand SQOOP, perform structural modifications using Map - Reduce, HIVE and analyze data using visualization/reporting tools.
• Hands on experience and Good Knowledge on real time data feeding platform-KAFKA, integration software like Talend,
• Excellent working exposure to Denodo Virtual Data Port, MS SQL Server, Informatica, DataStage, UNIX/Linux, Unix Shell Script, Crontab & Control-M
• Good Knowledge on NOSQL databases like MongoDB, HBase, and Cassandra.
• Experience in using Sqoop to import data and export data into HDFS from RDBMS.
• Solid understanding of open source monitoring tool: Cloudera Manager.
• Experience in using and understanding of Pig, Hive and HBase and Hive Built-in functions and Hive partitioning, bucketing and perform different types of joins on Hive tables and HDFS Designs, Daemons, HDFS high availability (HA).
• Over 3+ years of comprehensive IT experience in Big Data and Big Data Analytics, Hadoop, HDFS, Map Reduce, YARN, Hadoop Ecosystem and Shell Scripting.
• 5+ years of development experience using Java, J2EE, JSP, and Servlets.
• Highly capable of processing large sets of Structured, Semi-structured and Unstructured datasets and supporting Big Data applications.
• Hands on experience with Hadoop Ecosystem components like Map Reduce (Processing), HDFS (Storage), YARN, Sqoop, Pig, Hive, HBase, Oozie,and Zookeeper.
• Expertise in transferring data between a Hadoop ecosystem and structured data storage in an RDBMS such as MY SQL, Oracle, Teradata and DB2 using Sqoop.
• Expertise in moving large amounts of thelog, streaming event data and Transactional data using Flume.
• Experience in developing Map Reduce jobs in Java for data cleaning and preprocessing.
• Expertise in writing Pig Latin, HiveScripts and extended their functionality using User Defined Functions (UDF's).
• Expertise in handling structured arrangement of data within certain limits (Data Layout's) using Partitions and Bucketing in Hive.
• Experience working with Cloudera Hue Interface and Impala.
• Expertise in Object-oriented analysis and design (OOAD) like UML and use of various design patterns.
• Experience in Java, JSP, Servlets, EJB, Web Logic, Web Sphere, Hibernate, Spring JBoss, JDBC, RMI, Java Script, Ajax, JQuery, XML, and HTML.
• Fluent with the core Java concepts like I/O, Multi-threading, Exceptions, RegEx, Collections, Data-structures and Serialization.
• Performed unit testing using JUnit Testing Framework and Log4J to monitor the error logs.
• Expertise in working with transactional databases like Oracle, SQL Server, My SQL, and Db2.
• Expertise in developing SQL queries, Stored Procedures.
• Has very good development experience with Agile Methodology.
• Worked on streaming the data into HDFS from web servers using Flume
• Worked extensively with Sqoop for importing/exporting data between relational sources like SQL Server/MySQL and HDFS/Hive.
• Experience in importing the data from the MySQL into the HDFS using Sqoop.
• Experience in fine-tuning performance of Hive queries and Pig scripts.
• Experience in developing Map Reduce jobs in Java for data cleaning and preprocessing.
• Expertise in writing Pig Latin, HiveScripts and extended their functionality using User Defined Functions (UDF's).
• Expertise in handling structured arrangement of data within certain limits (Data Layout's) using Partitions and Bucketing in Hive.
• Expertise in preparing interactive Data Visualization's using Tableau Software from different sources.
• Knowledge ofNoSQL databases like HBase and Cassandra.
• Hands on experience developing workflows that execute Map Reduce, Sqoop, Pig, Hive and Shell scripts using Oozie.
• Experience in IT Service Delivery areas of Configuration, SLM, Availability, Change, Problem and IT Service Continuity Management
• Technically strong in IBM Mainframe technologies. Development experience in COBOL, DB2 and JCL.

TECHNICAL SKILLS

Operating System: Linux, UNIX,IOS, TinyOS, Sun Solaris, HP-UX, Windows 8, Windows 7, UNIX, Linux, CentOS, Ubuntu.

Scripts: UNIX Shell scripting, SQL and PL/SQL,JavaScript, JQuery and VB Script.

Database: SQL Server,Oracle, and MYSQL, HBase, MongoDB, Cassandra.

Debugging tools: Microsoft SQL Server Management Studio 2008, Business Intelligence Development Studio 2008, RAD, Subversion, BMS Remedy

Version Controller: SVN, GIT, CVS,Tpump,Mload,Fast Export.

Web Technologies: ASP.NET,ASP,HTML,DHTML,CSS,WebServices,XML,Java,HTML,CSS,JSP,Web Services,XML, JavaScript.

GUI Editors: Eclipse, Notepad++, Sublime Text, Brackets, Web storm, JBoss.

JavaScript Frameworks: MVC, Struts, Spring, Hibernate, JDBC, Struts.

Others: MS Office, Adobe Photoshop, DOM manipulations, Responsive Web Design, Karma, Grunt, Jasmine, Rally, Confluence, Jira.

Tools: Eclipse, Tableau, GIT, SVN, Concurrent versions system (CVS),RAD, Subversion, BMS Remedy, PUTTY, WinSCP, FileZilla, Service Now

PROFESSIONAL EXPERIENCE

Confidential, Los Angeles, CA

Hadoop and Bigdata Developer

Responsibilities:

• Involved in requirements gathering, analysis, and design. Involved in development and testing of the complete business solution.
• Analyzing the metadata as per the requirements
• Loaded the data in HDFS from thelocal system.
• Utilized the Denodo platform to create a virtual data layer via REST web services for both internal and external (cloud) clients.
• Writing the pig scripts for processing the data.
• Created hive table with schema and loaded the data using Sqoop.
• Writing queries in thehive to map the data resided in HDFS.
• DevelopingMapReduce scripts to count large number of records in HBase tables
• Working on different hive optimization and performance tuning techniques.
• Working on Ingestion of logs into Hadoop using Flume and Kafka
• Processing logs using spark streaming and loaded into hive tables
• Using Hive SerDe to read and write data in different formats.
• Involved in loading data from UNIX file system to HDFS.
• Responsible for building scalable distributed data solutions using Hadoop.
• Involved in loading data from theedge node to HDFS.
• Worked on POCs (Proof Of Concepts) for Data Virtualization with DENODO and CISCO.
• Involved in Design, Architecture, and Installation of Big Data and Hadoop ecosystem components.
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
• Created a virtualization layer using DENODO tool for EARMDB source
• Creating workflows using oozie
• Automated Hadoop jobs using Oozieschedule.
• Responsible for building scalable distributed data solutions using Hadoop.
• Involved in Data modeling sessions to develop models for Hive tables
• Imported and exported large sets of data into HDFS and vice-versa using Sqoop.
• Worked on Integrating Talend and SSIS with Hadoop and performed ETL operations.
• Installed Hive, Pig, Flume, Sqoop, and Oozie on the Hadoop cluster.
• Involved in Unit testing and documentation of the ETL Process
• Involved in Performance tuning at thesource, target, mappings, sessions, and system levels.
• Worked with NoSQL databases like HBase in creating tables to load large sets of semi structured data coming from various sources.
• Involved in building the ETL architecture and Source to Target mapping to load data into Data warehouse.
• Responsible for understanding the scope of the project and requirements gathering.
• Responsible for analyzing large data sets and derive customer usage patterns by developing new Map Reduce programs.
• Responsible for writing Struts action classes, Hibernate POJO classes and integrating Struts and Hibernate with Spring for processing business needs.
• Implemented Kafka Custom encoders for custom input format to load data into Kafka Partitions.
• Exporting of theresult set from HIVE to MySQL using Sqoop export tool for further processing.
• Working on different hive optimization and performance tuning techniques.

Environment: Java 1.2/1.3, Swing, Applet, Servlets, JSP, custom tags, JNDI, JDBC, XML, XSL, DTD, HTML, CSS, Java Script, Oracle, DB2, PL/SQL, WebLogic, JUnit, Log4J and CVS, Microsoft SQL Server, Management Studio 2008, Service Now, LINUX, Denodo

Confidential, Detroit, Michigan

Hadoop and Bigdata Developer

Responsibilities:

• Involved in Business requirements and Technical Validation sessions.
• Understanding existing business model, customer requirements and prepared technical design documents for various entities.
• Extensively used ETL methodology for supporting Data Extraction transformations and Loading process
• Created ETL mappings as well as incremental loads
• The Existing Fact and Dimensional tables, ODS were migrated from Informatica based ETL to subsequently data feeds and reports were generated out of them
• Used Informatica power center to create mappings, sessions, and workflows populating the data into dimensions and facts.
• Involved in Unit testing and documentation of the ETL Process.
• Migrated complex Map reduces programs into Spark RDD transformations, actions.
• Implemented KafkaHigh-level consumers to get data from Kafka partitions and move into HDFS.
• Worked on analyzing Hadoop cluster and different big data analytic tools including Map Reduce, Hive, and spark.
• Implemented Kafka Custom encoders for custom input format to load data into Kafka Partitions.
• Exporting of theresult set from HIVE to MySQL using Sqoop export tool for further processing.
• Evaluated the performance of Apache Spark in analyzing genomic data.
• Implemented Hive complex UDF's to execute business logic with Hive Queries.
• Implemented Impala for data analysis.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Load and transform large sets of structured, semi structured, and unstructured data with Map Reduce, Hive.
• Involved in loading data from LINUX file system, servers, Java web services using Kafka Producers, partitions.
• Evaluate usage of Oozie for Workflow Orchestration.
• Worked with NoSQL databases like HBase in creating tables to load large sets of semi structured data coming from various sources.
• Created partitioned tables in Hive, mentored analyst and test team for writing Hive Queries.
• Involved in acluster setup, monitoring, test benchmarks for results.
• Good understanding and related experience with Hadoop stack-internals, Hive, Pig and Map Reduce.
• Responsible for Business Analysis and Requirements Collection.
• Worked on Informatica Power Center tools- Designer, Repository Manager, Workflow Manager, and Workflow Monitor.
• Parsed high-level design specification to simple ETL coding and mapping standards.
• Designed and customized data models for Data warehouse supporting data from multiple sources in real time.
• Resolvedproduct complications at customer sites and funneled the insights to the development and deployment teams to adopt long term product development strategy with minimal roadblocks.

Environment: Hadoop, MapReduce, HDFS, Flume, HBase, Hive, Pig, TAC, Java, Rational Rose, Spring 2.0, Hibernate 3.2, Web Logic, Eclipse, SQL Server 2008.

Confidential, Minnesota

Hadoop and Bigdata Developer

Responsibilities:

• Experience in configuration, management, supporting and monitoring Hadoop cluster using Cloudera distribution.
• Worked in Agile scrum development model on analyzing Hadoop cluster and different Big Data analytic tools including Map Reduce, Pig, Hive, Flume, Oozie, and SQOOP.
• Configured Hadoop MapReduce, HDFS, developed multiple MapReduce jobs in Java for data cleaning and preprocessing.
• Loaded data into thecluster from dynamically generated files using Flume and from relational database management systems using Sqoop.
• Established custom MapReduce programs to analyze data and used Pig Latin to clean unwanted data.
• Involved in creating Hive tables, loading with data and writing Hive queries that will run internally in map reduce way.
• Involved in creating use case, class, sequence, package dependency diagrams using UML.
• Also involved in analysis and requirements gathering phase.
• Developed Server side code using Servlets, JSPs running on Apache Tomcat 3.0 and Enterprise Beans running on IBM Web Sphere Application Server.
• Developed web pages using HTML, JSP, DHTML and CSS.
• Used JavaScript for certain form validations, submissions, and other client side operations.
• Created Stateless Session Beans to communicate with the client.
• Created the database tables in Oracle 7i; created the required SQL queries and used JDBC to perform database operations.
• Developed activity, sequence and class diagrams using Unified Modeling Language and Rational Rose.
• Developed HTML and JSP pages using Struts.
• Developed Controller Servlets, Action and Form objects for theprocess of interacting with Oracle database and retrieving dynamic data.
• Responsible for coding SQL Statements and Stored procedures for back end communication using JDBC.
• Used Apache Log4J logging API to log errors and messages.
• Responsible for understanding the scope of the project and requirements gathering.
• Responsible for analyzing large data sets and derive customer usage patterns by developing new Map Reduce programs.
• Used Map Reduce to Index the large amount of data to easily access specific records.
• Load log data into HDFS using Flume. Worked extensively in creating Map Reduce jobs to power data for search and aggregation.
• Worked extensively with Sqoop for importing data from Oracle.
• Utilized Apache Hadoop ecosystem tools like HDFS, Hive, and Pig for large datasets analysis.
• Designed and developed UDF’S to extend the functionality in both PIG and HIVE.
• Created Partitioned Hive tables and worked on them using Hive.
• Developed scripts and batch jobs to schedule various Hadoop programs.
• Involved in the analysis & design of the application using Rational Rose.
• Responsible for configuring and deploying the builds on Web logic App Server.

Environment: Hadoop - CDH 5.0.2, PIG, Hive, Apache Sqoop, Oozie, HBase, Zoo keeper, CDH 4.0, HDFS, Hive, Java, Sqoop, Cloudera CDH4, MySQL, Tableau, XML, HTML, Sharepoint, FileZilla, PUTTY, UNIX.

Confidential, Pennsylvania

Hadoop and Bigdata Developer

Responsibilities:

• Responsible for preparation of Technical Design document to meet the requirements of functional requirement documents.
• Responsible for development of Code for application changes as required/specified by technical design documents.
• Involved in PGS/UTC and Code reviews, code walk troughs and Unit testing using the test data prepared by CIGNA.
• Involved in Knowledge Acquisition/ Knowledge Transition of the Proclaim Preprocessing Domain to the newly inducted Team members and attended the DP and weekly Status meetings.
• Assist developers in the technical design phase, construction and unit-testing phase.
• Have done online Transactions in the IMS/DC region. Tested the programs and debugged using Xpeditor Tool.
• Used File Aid for Browse, Edit View Record Layout, Reformat and Compare records
• Coded the IMS/DC functions, IMS/DC online programs. Developed Screens using macros BasicMappingSupport, physical, symbolic maps and I/O operations.
• Involved in gathering user requirements and followed by analysis and design.
• Involved in Developing JSP's, action classes, form beans, response beans, EJB's.
• Developed Struts configuration files and action classes the Transactional Model to handle many requests.
• Participated in developing helper classes which interact with middleware stubs.
• Extensively used XML to code configuration files.
• Developed PL/SQL stored procedures, triggers.
• Performed functional, integration, system and validation testing.
• Responsible to manage data coming from different sources.
• Involved in creating Hive Tables, loading data and writing hive queries.
• Involved in scheduling Oozie workflow engine to run jobs automatically.
• Implemented No SQL database like HBase for storing and processing different formats of data.
• Involved in Testing and coordination with business in User testing.
• Involved in Unit testing and delivered Unit test plans and results in documents.
• Involved in unit testing and system testing and also responsible for preparing test scripts for the system testing.
• Responsible for packaging and deploying components into the WebSphere.
• Developed backend components, DBScripts for the backend communication.
• Responsible for performance tuning of the product and eliminating memory leakages in the product.
• Developed Controller Servlets, Action and Form objects for theprocess of interacting with Oracle database and retrieving dynamic data.
• Responsible for writing Struts action classes, Hibernate POJO classes and integrating Struts and Hibernate with Spring for processing business needs.

Environment: Java, J2EE, JDBC, Struts, Servlets, JSP, JavaScript, HTML, SQL. Hibernate, Eclipse, Apache POI, CSS, JDBC, XML, PL/SQL, SQL, UNIX, CSS, Eclipse, Net Beans, HTML, JNDI, Junit, Log4J.

Confidential

Hadoop and Bigdata Developer

Responsibilities:

• Involved in creating use case, class, sequence, package dependency diagrams using UML.
• Also involved in analysis and requirements gathering phase.
• Developed Server side code using Servlets, JSPs running on Apache Tomcat 3.0 and Enterprise Beans running on IBM Web Sphere Application Server.
• Developed web pages using HTML, JSP, DHTML and CSS.
• Used JavaScript for certain form validations, submissions, and other client side operations.
• Created Stateless Session Beans to communicate with the client.
• Created the database tables in Oracle 7i; created the required SQL queries and used JDBC to perform database operations.
• Involved in design and implementation of web tier using Servlets and JSP.
• Used Apache POI for Excel files reading.
• Developed the user interface using JSP and Java Script to view all online trading transactions.
• Designed and developed Data Access Objects (DAO) to access the database.
• Used DAO Factory and value object design patterns to organize and integrate the JAVA Objects.
• Coded Java Server Pages for the Dynamic front end content that use Servlets and EJBs.
• Coded HTML pages using CSS for static content generation with JavaScript for validations.
• Used JDBC API to connect to the database and carry out database operations.
• Used JSP and JSTL Tag Libraries for developing User Interface components.
• Monitored the servers and Linux scripts regularly and performed troubleshooting step
• Worked with SVN Version control system to track various aspects of the project.
• Worked with Jenkins for continuous integrations
• Used Asynchronous JavaScript and XML (AJAX) for better and faster interactive Front-End.
• Responsible for configuring and deploying the builds on Web logic App Server.
• Used Maven for dependency management and for building the application.
• Developed Spring AOP aspects to address cross cutting concerns.
• Developed middleware components using spring and Hibernate. Integrated spring with hibernate for spring to manage Hibernate session.
• Experience in providing Logging, Error handling by using Event Handler, and Custom Logging for SSIS Packages.
• Resolvedproduct complications at customer sites and funneled the insights to the development and deployment teams to adopt long term product development strategy with minimal roadblocks.
• Convinced business users and analysts with alternative solutions that are more robust and simpler to implement from atechnical perspective while satisfying the functional requirements from the business perspective.
• Applied design patterns and OO design conceptsto improve the existing Java/JEE based code base.

Environment: Cloudera, Hive, Pig, Hive Bucketing, Hive partitioning, Linux, Oozie, Map-Reduce, Sqoop, HBase, Shell Scripting,MapReduce, HDFS, Hive, HBase, Oozie, SQL, PL/DSQL, Eclipse, Oracle, Windows XP, HTML, CSS, JavaScript, and XML.

Confidential

Hadoop and Bigdata Developer

Responsibilities:

• Responsible for coding SQL Statements and Stored procedures for back end communication using JDBC.
• Used Apache Log4J logging API to log errors and messages.
• Responsible for understanding the scope of the project and requirements gathering.
• Responsible for analyzing large data sets and derive customer usage patterns by developing new Map Reduce programs.
• Used Map Reduce to Index the large amount of data to easily access specific records.
• Load log data into HDFS using Flume. Worked extensively in creating Map Reduce jobs to power data for search and aggregation.
• Worked extensively with Sqoop for importing data from Oracle.
• Utilized Apache Hadoop ecosystem tools like HDFS, Hive, and Pig for large datasets analysis.
• Designed and developed UDF’S to extend the functionality in both PIG and HIVE.
• Used JavaScript for certain form validations, submissions, and other client side operations.
• Created Stateless Session Beans to communicate with the client.
• Created the database tables in Oracle 7i; created the required SQL queries and used JDBC to perform database operations.
• Involved in design and implementation of web tier using Servlets and JSP.
• Used Apache POI for Excel files reading.
• Implemented KafkaHigh-level consumers to get data from Kafka partitions and move into HDFS.
• Worked on analyzing Hadoop cluster and different big data analytic tools including Map Reduce, Hive, and spark.
• Implemented Kafka Custom encoders for custom input format to load data into Kafka Partitions.
• Exporting of theresult set from HIVE to MySQL using Sqoop export tool for further processing.
• Evaluated the performance of Apache Spark in analyzing genomic data.
• Implemented Hive complex UDF's to execute business logic with Hive Queries.
• Implemented Impala for data analysis.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Involved in Design, Architecture, and Installation of Big Data and Hadoop ecosystem components.
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
• Creating workflows using oozie
• Automated Hadoopjobs using Ooziescheduler.
• DevelopingMapReduce scripts to count large number of records in HBase tables
• Working on Ingestion of logs into Hadoop using Flume and Kafka.

Environment: COBOL, JCL, DB2, VSAM, CICS, FILE-AID, ENDEVOR, OS/390, XPEDITOR, IDCAMS, CDH 4.0, HDFS, Hive, Java, Sqoop, Cloudera CDH4, MySQL