PROFESSIONAL SUMMARY:

• Over 8 years of IT experience in various domains with Hadoop Ecosystems and Java J2EE technologies.
• Over 5 years of hands on experience Technical Designing and Developing BIG DATA solutions using Hadoop Ecosystems (HDFS, MapReduce, Pig, Hive, Sqoop, Hbase, Avro and Oozie scheduler).
• Good understanding of the mapreduce framework architectures (MRV1 & YARN Architecture).
• Good Knowledge and understanding of Hadoop Architecture and various components in Hadoop ecosystems - HDFS, Map Reduce, Pig, Sqoop and Hive
• Developed various Map Reduce applications to perform ETL workloads on terabytes of data.
• Hands on experience in cleansing semi-structured and unstructured data using Pig Latin scripts
• Good working knowledge in creating Hive tables and worked using HQL for data analysis to meet the business requirements.
• Experience in managing and reviewing Hadoop log files.
• Experience in handling multiple relational databases: NoSQL and HBase.
• Responsible to manage data coming from different sources and involved in HDFS maintenance and loading of structured and unstructured data.
• Experience in importing and exporting the data using Sqoop from HDFS to Relational Database systems/mainframe and vice-versa.
• Experience in working with flume to load the log data from multiple sources directly into HDFS
• Experience in scheduling time driven and data driven Oozie workflows.

TECHNICAL SKILLS:

Hadoop/Big Data platform: HDFS, Map Reduce, HBase, Hive, Pig, Oozie, Zookeeper, Flume, Sqoop, Apache Avro, Kafka.

Hadoop distribution: Hadoop, Cloudera

Programming languages: Java, Pig Latin

Operating Systems: Windows, Linux (Ubuntu)

Data Storage and Data Base: NoSQL, HBase

Designing Tools: Eclipse

Java Technologies: JSP, Servlets, Junit, Spring,Hibernate

Web Technologies: XML, HTML, JavaScript, JVM, JQuery, JSON

Linux Experience: System Administration Tools, Puppet, Apache

Web Services: Web Service (RESTful and SOAP)

Frame Works: Jakarta Struts 1.x, Spring 2.x

Development methodologies: Agile, Waterfall

Logging Tools: Log4j

Application / Web Servers: BEA Web Logic, Apache Tomcat,Websphere

Messaging Services: ActiveMQ, Kafka,JMS

Version Tools: Git, SVN and CVS

Analytics: Tableau, SPSS, SAS EM and SAS JMP

PROFESSIONAL EXPERIENCE:

Confidential, Peoria, Illinois

Hadoop Developer

Responsibilities:

• The main purpose of this project is to store the data in Hadoop cluster and extract the data using hive and pig Latin for processing according to the user requirements
• Involved in requirement gathering phase of the SDLC and helped team by breaking up the complete project into modules with the help of my team lead.
• Installed and configured Hadoop and Hadoop stack on a 4 node cluster.
• Developed Map Reduce programs to parse the raw data, populate staging tables and store the refined data in partitioned tables.
• Developed Map reduce programs for Third Party files to analyse data i.e Funds sold by a parent company to subsequent chain of companies.
• Involved in data ingestion into HDFS using Sqoop and Flume from variety of sources.
• Moved all flat data files generated from various sources to HDFS for further processing.
• Responsible for managing data from various sources.
• Worked on Kafka to produce the streamed data into topics and consumed that data.
• Got good experience with NoSQL database HBase.
• Worked with NoSQL databases like HBase in creating HBase tables to load large sets of semi structured data coming from various sources.
• Installed and configured Hive and also wrote Hive UDFs that helped spot market trends.
• Involved in loading data from UNIX file system to HDFS.
• Involved in creating Hive tables, loading the data using it and in writing Hive queries to analyze the data.
• Worked extensively on creating Oozie workflows for scheduling different jobs of hive, map reduce and shell scripts.
• Used Master data management(MDM) tools to provide the dimensions for big data facts, as a slew of big data projects highlight a link to data governance and quality.
• ETL tools are used to transform data into the format required by data warehouses.

Environment: HDFS, Pig, Hive, HBase, Sqoop, Oozie, flume, Kafka, Linux Shell Scripting.MR jobs, Master data management tools.

Confidential, Charlotte, North Carolina

Hadoop Developer

Responsibilities:

• Imported data from our relational data stores to Hadoop using Sqoop.
• Created various Mapreduce jobs for performing ETL transformations on the transactional and application specific data sources.
• Wrote PIG scripts and executed by using Grunt shell.
• Worked on the conversion of existing MapReduce batch applications for better performance.
• Big data analysis using Pig and User defined functions (UDF).
• Worked on loading tables to Impala for faster retrieval using different file formats.
• The system was initially developed using Java. The Java filtering program was restructured to have business rule engine in a jar that can be called from both java and Hadoop.
• Created Reports and Dashboards using structured and unstructured data.
• Upgrade operating system and/or Hadoop distribution as and when new versions released by using Puppet.
• Performed joins, group by and other operations in MapReduce by using Java and PIG.
• Processed the output from PIG, Hive and formatted it before sending to the Hadoop output file.
• Used HIVE definition to map the output file to tables.
• Setup and benchmarked Hadoop/HBase clusters for internal use
• Wrote data ingesters and map reduce programs
• Reviewed the HDFS usage and system design for future scalability and fault-tolerance
• Wrote MapReduce/HBase jobs
• Worked with HBase, NOSQL database.

Environment: Apache Hadoop 2.x, Map Reduce, HDFS, Hive, Pig, Hbase, Sqoop, Flume, Linux, Java 7, Eclipse, NOSQL.

Confidential, Salt lake City, Utah

Hadoop Developer

Responsibilities:

• Developed Big Data Solutions that enabled the business and technology teams to make data-driven decisions on the best ways to acquire customers and provide them business solutions.
• Installed and configured Apache Hadoop, Hive, and HBase.
• Worked on Hortonworks cluster, which was used to process the big data.
• Developed multiple map reduce jobs in java for data cleaning and pre-processing.
• Sqoop was used to pull data into Hadoop distributed file system from RDBMS and vice versa
• Defined workflows using Oozie.
• Used Hive to create partitions on hive tables and analyzes this data to compute various metrics for reporting.
• Created Data model for Hive tables
• Good Experience in managing and reviewing Hadoop log files
• Used Pig as ETL tool to do transformations, joins and pre-aggregations before loading data onto HDFS.
• Worked on large sets of structured, semi structured and unstructured data
• Responsible to manage data coming from different sources
• Installed and configured Hive and also developed Hive UDFs to extend core functionality of hive
• Responsible for loading data from UNIX file systems to HDFS.

Environment: Apache Hadoop 2.x, MapReduce, HDFS, Hive, HBase, Pig, Oozie,Linux, Java 7, Eclipse.

Confidential, Bridgewater, NJ

Sr.Java Developer

Responsibilities:

• Full life cycle experience including requirements analysis, high level design, detailed design, UMLs, data model design, coding, testing and creation of functional and technical design documentation.
• Used Spring Framework for MVC architecture with Hibernate to implement DAO code and also used Web Services to interact other modules and integration testing.
• Developed and implemented GUI functionality using JSP, JSTL, Tiles and AJAX.
• Designed database and involved in developing SQL Scripts.
• Used SQL navigator as a and involved in testing the application.
• Implementing the Design Patterns like MVC-2, Front Controller, Composite view and all Struts framework design patterns to improve the performance.
• Used Clear case, and also subversion for maintaining the source version control.
• Wrote Ant scripts to automate the builds and installation of modules.
• Involved in writing Test plans and conducted Unit Tests using JUnit.
• Used Log4j for logging statements during development.
• Design and implementation of log data indexing and search module, and optimization for performance and accuracy. To provide a full text search capability for archived log data, utilizing Apache Lucene library.
• Involved in the testing and integrating of the program at the module level.
• Worked with production support team in debugging and fixing various production issues.

Environment: s: Java1.5,AJAX,XML,Spring3.0,Hibernate2.0,Struts1.2,Webservices,Websphere7.0,Junit,Oracle10g,SQL, PL/SQL, log4j, RAD 7.0/7.5, Clear case, Unix, HTML, CSS, Java script.

Confidential

Java Developer

Responsibilities:

• Worked with the business community to define business requirements and analyze the possible technical solutions.
• Requirement gathering, Business Process flow, Business Process Modeling and Business Analysis.
• Extensively used UML and Rational Rose for designing to develop various use cases, class diagrams and sequence diagrams.
• Used JavaScript for client-side validations, and AJAX to create interactive front-end GUI.
• Developed application using Spring MVC architecture.
• Developed custom tags for table utility component
• Used various Java, J2EE APIs including JDBC, XML, Servlets, and JSP.
• Designed and implemented the UI using Java, HTML, JSP and JavaScript.
• Designed and developed web pages using Servlets and JSPs and also used XML/XSL/XSLT as repository.
• Involved in Java application testing and maintenance in development and production.
• Involved in developing the customer form data tables. Maintaining the customer support and customer data from database tables in MySQL database.

Environment: JDK 1.3, J2EE, JDBC, Servlets, JSP, XML, XSL, CSS, HTML, DHTML, JavaScript, UML, Eclipse 3.0, Tomcat 4.1, MySQL.