SUMMARY

• 7+ years of extensive IT experience as 70% Development and 30% administrator in Analysis, Design, Development, Implementation and Testing of software applications which includes 3+ years of experience in Big Data using Hadoop, Hive, Pig, Sqoop and MapReduce programming.
• Working experience in MapReduce programming model and Hadoop Distributed File Systems (HDFS).
• Hands on experience in major components of Hadoop Ecosystem like Flume, Hbase, Zookeeper, Oozie, Hive, Sqoop, PIG and YARN (MR2).
• Developed scripts and numerous batch process jobs to schedule various Hadoop programs.
• Experience in Amazon, Cloudera and Hortonworks Hadoop distribution.
• Worked on importing and exporting data from different databases like Oracle, MySQL, and SQL server into HDFS using sqoop.
• Experience in analyzing data using HiveQL, PIG Latin and MapReduce programs in java.
• Strong experience in collecting and storing streaming data like log data, twitter data into HDFS using Apache Flume.
• Experienced in working with MapReduce Design patterns to solve complex MapReduce programs.
• Wrote Hive and PIG queries for data analysis to meet the business requirements.
• Involved in creating tables, partitioning, bucketing and creating UDF’s in Hive.
• Implemented join operations and involved in writing data transformation using PIG Latin.
• Extensive knowledge in NoSQL databases like Hbase and Cassandra.
• Experienced in using CRUD operations using Hbase Java client API and Rest API.
• Good knowledge with Oozie Workflow engine to automate and parallelize Hadoop MR, Hive and PIG jobs.
• Excellent java development skills using J2EE Frameworks like spring, Hibernate, EJB’s and Web Services.

TECHNICAL SKILLS

Big Data Platforms: Cloudera, Big Data, Hadoop, Yarn, Map Reduce, PIG, HIVE, Storm, Kafka, Impala, FLUME and SPARK

Languages: Java, C++

Databases: Oracle, MySQL, SQL Server

No SQL Databases: Hbase, Cassandra, MongoDB

Job Scheduling framework: Auto Sys, Quartz Scheduler

Operating Systems: Linux, Unix, Windows 7, Windows 8, XP, Windows vista

Web Technologies: HTML, XHTML, Java Script

Project management: SDLC models, JAD, Project scheduling, Cost and Time estimation

Data Modelling tools: MS Visio, Rational Rose

PROFESSIONAL EXPERIENCE

Confidential, New Brunswick, NJ

Sr. Hadoop Developer/Administrator

Responsibilities:

• Involved in requirement analysis, design, coding and implementation.
• Installation, configuration and administration experience in big data platform Cloudera CDH.
• Responsible for building scalable distributed data solutions using Hadoop Cloudera.
• Responsible for Cluster maintenance, commissioning and decommissioning Data nodes, Cluster Monitoring, Troubleshooting, Manage and review data backups, Manage & reviewHadoop Log files
• Processed data into HDFS by developing solutions and analyzed the data using Map Reduce, PIG, and Hive to produce summary results from Hadoop to downstream systems.
• Used Sqoop to import the data from Hadoop Distributed File System (HDFS) to RDBMS.
• Established custom Map Reduce programs in order to analyze data and used Pig Latin to clean unwanted data.
• Configured various property files like core - site.xml, hdfs-site.xml, mapred-site.xml based upon the job requirement
• Worked on various performance optimizations like using distributed cache for small datasets, partition, Bucketing and Map side joins.
• Involved in creating Hive tables and applied those HQL on the tables for data validation.
• Responsible for Installation and configuration of Hive, Pig, Hbase and Sqoop on theHadoop cluster.
• Involved in loading and transforming large sets of Structured, Semi-Structured and Unstructured data and analyzed them by running Hive queries and Pig scripts.
• Used Zookeeper to manage coordination among the clusters.
• Worked with Impala to pull the data from Hive tables.
• Installed Oozie workflow engine to run multiple Hive and Pig jobs which run independently with time and data availability.
• Involved in Cluster maintenance, Cluster Monitoring and Troubleshooting, Manage and review data backups and log files.

Environment: HDFS, Hadoop, Pig, Hive, Sqoop, Flume, Map Reduce, Oozie, Mongo DB, Java 6/7, Oracle 10g, Sub Version, Toad, UNIX Shell Scripting, SOAP, REST services, Oracle 10g, Agile Methodology, JIRA, Auto Sys.

Confidential, Windsor, CT

Sr. Hadoop Developer

Responsibilities:

• Created Hive tables and loaded retail transaction data from Teradata using Sqoop.
• Loaded home mortgage data from the existing DWH tables (SQL Server) to HDFS using Sqoop.
• Wrote Hive queries to have a consolidated view of the mortgage and retail data.
• Data is loaded back to the Teradata for the BASEL reporting and for the business users to analyze and visualize the data using Datameer.
• Orchestrated hundreds of Sqoop scripts, Pig scripts, Hive queries using oozie workflow and sub-workflows.
• Loaded the files from mainframes to Hadoop and files were converted to ASCII format.
• Developed Pig Latin scripts for replacing the existing home loans legacy process to the Hadoop and the data is fed to retail legacy mainframes systems.
• Lead the QA effort of the customer facing channels for the CSS program
• Managed a team of 2-4 manual and automation testers throughout the program
• Reviewed business and functional requirements and provided feedback to the business analysts and business owners for more clarity
• Developed the test package and clearly defined the scope of testing
• Demonstrated a clear understanding of different systems and applied the skills in developing integration testing strategy
• Successfully completed test execution for all small to extra-large projects with 99% success rate
• Developed test scripts using QTP automation tool and founder of the JetBlue QA automation framework which scored us HP award for Automation excellence.
• Developed a thorough understanding of all shopping and self-service channels via CSS program.
• Worked as Quality center admin for the CSS program
• Migrated Quality center application QC 8.0

Environment: Hadoop Distributed File System (HDFS), MapReduce, Hive, Pig, Sqoop, Kafka, Java 7.0, XML, WSDL, SOAP, Web services, Oracle/Informix, Log4J, Junit, SVN (Apache subversion), maven and Oozie.

Confidential, Pittsburgh, PA

Java/Hadoop Developer

Responsibilities:

• Involved in gathering functional requirements that are involved in order to fulfill Discovers specific requirements for the online Claims app and developed wireframes.
• Project is implemented in SCRUM and Agile methodology.
• Analyzed Hadoop clusters and other analytical tools used in big data like Hive, Pig and databases like Hbase.
• Extracted feeds from social media sites such as Facebook and Twitter using Flume.
• Used Sqoop extensively to ingest data from various source systems into HDFS.
• Written Hive queries for data analysis to meet the business requirements.
• Created Hive tables and worked on them using Hive QL.
• Installed cluster, worked on commissioning & decommissioning of Name node.
• Used Spring Integration (SI) to expose some services of our application for other applications in the company use.
• Used SOAP UI to test the SOAP web services.
• Managed, created and altered Databases, Tables, views and indexes with business rules using T-SQL.
• Created complex Stored Procedures, Triggers and User Defined Functions to support the front-end application.
• Implemented the error handling in the Stored Procedures and also modified already existing stored procedures, triggers, views and indexing depending on the requirement.
• Involved in Master Data Management.

Environment: Java, J2EE, JDBC, Oracle 9i,Log4j, Web Services, SOA, SOAP, WSDL, Maven, TC Server, Hadoop, HDFS, Pig 0.10, Hive, Map Reduce, HBase, Sqoop, Java Eclipse, SQL Server, Shell Scripting, MS SQL Server 2005/2008 R2, Visual Studio 2010, TFS 2010, MS Office, MS Excel.

Confidential

Java Developer

Responsibilities:

• Created UML class diagrams that depict the code's design and its compliance with the functional requirements.
• Used J2EE design patterns for the middle tier development.
• Developed EJB's in Web Logic for handling business process, database access and asynchronous messaging.
• Used Java Mail notification mechanism to send confirmation email to customers about scheduled payments.
• Developed Message-Driven beans in collaboration with Java Messaging Service (JMS) to communicate with merchant systems.
• Also involved in writing JSP's/JavaScript and Servlets to generate dynamic web pages and web content.
• Wrote stored procedures and Triggers using PL/SQL.
• Involved in building and parsing XML documents using JAX parser.
• Deployed the application on Tomcat Application Server.
• Experience in implementing Web Services and XML/HTTP technologies.
• Created UNIX shell and Perl utilities for testing, data parsing and manipulation.
• Used Log4J for log file generation and maintenance.
• Wrote JUnit test cases for testing.

Environment: Java, JDBC, Servlets, JSP, Struts, Eclipse, Oracle 9i, CVS, JavaScript, Log4J, J2EE, JDK6, Java Script, EJB, Web Services, SOAP, WSDL, Application Server, Oracle 10g/11g, SQL, Log4j, XML, XPATH, XSD, HTML, TFS, JUnit, CSS.