SUMMARY:

• Around 8 years of experience in software development, 3+ years of experience in all phases of Hadoop Technologies.
• Excellent understanding in depth knowledge of Hadoop architecture and various components such as HDFS, Map Reduce programming and other Ecosystem components.
• Experienced on major Hadoop ecosystem projects such as PIG, HIVE and HBASE.
• Good working experience using Sqoop to import data into HDFS from RDBMS and vice - versa
• Participated in requirement analysis, reviews and working sessions to understand the requirements and systems design
• Experience in developing MapReduce Programs using Apache Hadoop for analyzing the big data as per the requirement.
• Experience in developing solutions to analyze large data sets efficiently
• Knowledge on SPARK, KAFKA, STORM
• Good knowledge of Nosql database HBASE, Cassandra and Mongodb
• Good knowledge on scripting languages like PYTHON, SCALA
• Developed Spark scripts by using Scala Shell commands as per the requirement.
• Written Python unit tests for testing REST API Web Services
• Extending Hive and Pig core functionality by writing custom UDFs
• Experienced in javascript, good understanding of AngularJS,JS Functions, JS Objects, JS HTML, DOM, JS Library (jQuery) etc
• Performed Web Service Testing experience using SOAP U
• Experience in SQL programming including SQL queries using stored procedures and Triggers in Oracle, SQL Server
• Hands-on development and implementation experience in Big Data Management Platform (BMP) using Hadoop, MapReduce, Hive and other Hadoop related eco-systems as a Data Storage and Retrieval systems
• Experience in managing and reviewing Hadoop Audit Files
• Experience in installation, configuration, support, maintenance of Cloudera’s Hadoop platform along with CDH4&5 clusters
• Design and Built database in Hadoop HIVE and Redshift.
• Enabled High Availability using QJM and NFS to avoid single point failure
• Experience in automating the Hadoop Installation, Configuration and Maintaining the cluster using tools Cloudera Manager, Ambari
• Used Maven extensively for building MapReduce jar files and deployed it to Amazon Web Services(AWS) using EC2 virtual Servers in the cloud and Experience in build scripts to do continuous integrations systems like Jenkin
• Experienced in Cluster building and Cluster monitoring and troubleshooting
• Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop MapReduce, HDFS, HBase, Oozie, Hive, Sqoop and Pig.
• Strong problem solving skills, good communication, interpersonal skills and a good team player
• Have the motivation to take independent responsibility as well as ability to contribute and be a productive team member
• Identify test cases to automate. Automate those test cases using Selenium WebDriver, Cucumber and Java technologies Junit in JENKINS.
• Experience in understanding Big Data business requirements and providing them Hadoop based solutions.
• Basic knowledge in application design using Unified Modeling Language (UML), Sequence diagrams, Case diagrams, Entity Relationship Diagrams (ERD) and Data Flow Diagrams (DFD).
• Experience in analyzing existing Hadoop cluster, understanding the performance bottlenecks and providing the performance tuning solutions accordingly.
• Extensive experience in working with the Customers to gather required information to analyze, debug and provide data fix or code fix for technical problems, build service patch for each version release and unit testing, integration testing, User Acceptance testing and system testing and providing Technical Solution documents for the Users
• Comprehensive knowledge of Software Development Life Cycle coupled with excellent communication skills. Strong technical and interpersonal skills combined with great commitment towards meeting deadlines.

TECHNICAL SKILLS:

Hadoop/Big Data Technologies: Hadoop, HDFS, Hive, Mapreduce, Pig, Sqoop, Flume,Yarn, Zookeeper, HBase NoSQL, Oozie, Impala, Sentry, Navigator

Programming Languages: Java, C, Python

Operating Systems: UNIX, Windows, LINUX

Databases: MySQL, MSSQL, PostgreSQL & Oracle

Tools / Version Control: GIT, JIRA, SVN, MRUnit, Eclipse

PROFESSIONAL EXPERIENCE:

Confidential, Vernon Hills, IL

Hadoop Developer

Responsibilities:

• Developed Big Data Solutions that enabled the business and technology teams to make data-driven decisions on the best ways to acquire customers and provide them business solutions.
• Involved in installing, configuring and managing Hadoop Ecosystem components like Hive, Pig, Sqoop and Flume.
• Migrated the existing data to Hadoop from RDBMS using Sqoop for processing the data.
• Responsible for loading unstructured and semi-structured data into Hadoop cluster coming from different sources using Flume and managing.
• Developed MapReduce programs to cleanse and parse data in HDFS obtained from various data sources and to perform joins on the Map side using distributed cache.
• Responsible for creating Hive tables, loading the structured data resulted from Mapreduce jobs into the tables and writing Hive Queries to further analyze the data
• Wrote SQL queries and performed Back-End Testing for data validation to check the data integrity during migration from bac-kend to front-end.
• Used Hive data warehouse tool to analyze the data in HDFS and developed Hive queries.
• Created internal and external tables with properly defined static and dynamic partitions for efficiency.
• Worked on setting up Pig, Hive, Redshift and Hbase on multiple nodes and developed using Pig, Hive, Hbase and MapReduce.
• Used the RegEx, JSON and Avro SerDe’s for serialization and de-serialization packaged with Hive to parse the contents of streamed log data.
• Implemented Hive custom UDF’s to achieve comprehensive data analysis.
• Wrote MRUnit tests for unit testing the MapReduce jobs.
• Wrote test cases in JUnit for unit testing of classes
• Exported the business required information to RDBMS using Sqoop to make the data available for BI team to generate reports based on data.
• Implemented daily workflow for extraction, processing and analysis of data with Oozie.
• Responsible for troubleshooting MapReduce jobs by reviewing the log files.
• As a POC we have installed Cassandra on the Amazon EC2
• Created tables, inserted data and executed various Cassandra Query language (CQL 3) commands on tables using cqlsh

Environment: Hadoop, Hive, MapReduce, Oozie,Oracle 11g, Sqoop, Flume, Core Java, Eclipse,Hue.

Confidential, Manhattan, NY

Hadoop Developer

Responsibilities:

• Developed MapReduce programs to parse the raw data, populate staging tables and store the refined data in partitioned tables
• Installed & maintained cloudera Hadoop distribution
• Installed and configured Flume, Hive, Pig, Sqoop and Oozie on the Hadoop cluster.
• Involved in loading the data from Linux file system to HDFS.
• Implemented Mapreduce programs on log data to transform into structured way to find user information.
• Performed performance tuning and troubleshooting of MapReduce jobs by analyzing and reviewing Hadoop log files.
• Exported the analyzed data to the relational databases using sqoop for virtualization and to generate reports for the BI team
• Monitored workload, job performance and capacity planning using Cloudera Manager.
• Installed Oozie workflow engine to run multiple Mapreduce, Hive and Pig jobs.
• Responsible for creating Hive tables, loading the structured data resulted from MapReduce jobs into the tables and writing hive queries to further analyze the logs to identify issues and behavioral patterns.
• Imported data frequently from MySQL to HDFS using Sqoop.
• Supported operations team in Hadoop cluster maintenance activities including commissioning and decommissioning nodes and upgrades.
• Used ETL tool, Talend to do transformations, event joins, filter and some pre-aggregations
• Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts.
• Used Tableau for visualizing and to generate reports.

Environment: Hadoop,Cloudera, MapReduce, Hive, Sqoop, Flume,Talend, Python, MS-SQL Server, Tableau.

Confidential, IN

Hadoop Developer

Responsibilities:

• Installation of Hadoop on distributed environment and its configuration.
• Maintains HBASE for the customer details, product details, Agent details and some other Misc.
• Uses Bloom filters to stop bot users in order to provide fraud detection while paying premium.
• Uses Flume to get the data of the Project Web Server Logs into HDFS.
• Upon the HDFS data, Hcatalog creates the schema and made the table schema as shared.
• Upon the Hcatalog tables we use pig and hive to analyze the data and also create schema for the hbase table in Hive.
• Behind all this we do sentimental analysis to know how public feels about our products now and then how the opinions change over time, like before and after the Product launch.
• Visualize the HDFS data to customer using BI tool (Excel 2013) with the help of Hive ODBC Driver.
• Have experience in Working with Hortonworks Sandbox HUE.
• Sqoop some Master tables from sql server into HDFS, Hbase and Hive.

Environment: Hadoop framework, Hortonworks, Hue Mapreduce, HDFS, Pig, Hive, HBase, Sqoop, Flume, Oozie.

Confidential, CT

Java/J2EE Developer

Responsibilities:

• Involved in various phases of Software Development Life Cycle (SDLC) as design development and unit testing.
• Developed and deployed UI layer logics of sites using JSP, XML, JavaScript, HTML/DHTML, and Ajax.cc
• Designed different design specifications for application development that includes front-end, back-end using design patterns.
• Developed prototype test screens in HTML and JavaScript.
• Involved in developing JSP for client data presentation and, data validation on the client side with in the forms.
• Developed the application by using the Spring MVC framework.
• Collection framework used to transfer objects between the different layers of the application.
• Developed data mapping to create a communication bridge between various application interfaces using XML, and XSL.
• Spring IOC being used to inject the parameter values for the Dynamic parameters.
• Developed Junit testing framework for Unit level testing.
• Actively involved in code review and bug fixing for improving the performance.
• Documented application for its functionality and its enhanced features.
• Created connection through JDBC and used JDBC statements to call stored procedures.
• Created UML diagrams like use cases, class diagrams, interaction diagrams, and activity diagrams.
• Extensively worked on User Interface for few modules using JSPs, JavaScript and Ajax.
• Created Business Logic using Servlets, POJO's and deployed them on Weblogic server.
• Wrote complex SQL queries and stored procedures.
• Developed the XML Schema and Web services for the data maintenance and structures.
• Implemented the Web Service client for the login authentication, credit reports and applicant information using Apache Axis2 Web Service.
• Designed the logical and physical data model, generated DDL scripts, and wrote DML scripts for Oracle 10g database.
• Used Hibernate ORM framework with Spring framework for data persistence and transaction management.
• Used struts validation framework for form level validation.
• Wrote test cases in JUnit for unit testing of classes.
• Involved in creating templates and screens in HTML and JavaScript.
• Involved in integrating Web Services using SOAP.

Environment: Spring MVC, Oracle 11g J2EE, Java, JDBC, Servlets, JSP, XML, CSS, HTML, JavaScript 1.2, Junit, Apache Tomcat, Mysql Server 2008.

Confidential

Java Programmer

Responsibilities:

• Worked with requirement analysis team to gather software requirements for application development.
• Designed UML and entity relational diagrams for the process flow and database design.
• Developed java programs to implement the computational logic for the web applications.
• Implemented model view controller architecture with the help of HTML5, JSPs, Servlets and Java.
• Designed and implemented the database server as backend for web application
• Provided support and maintenance after deploying the web application.
• Designed static web user interface with html and CSS
• Administered entire end to end flow of the web application.
• Developed custom packages to connect to standard data sources and retrieve data efficiently eliminating the need for each team to rewrite the same set of code multiple times.
• Worked on JavaScript, jQuery for data validation on client side.
• Worked on product deployment, documentation and support.
• Involved in structuring Wiki and Forums for product documentation
• Involved in R&D, set up and designing Mediawiki, PHP and Joomla content management systems.
• Worked on incorporating LDAP service and Single sign on for the CMS web portal.
• Maintained the customer support portal.
• Worked with Java Messaging Service(JMS)

Environment: Java, Servlets, JSP, JavaScript, HTML, PHP, CSS, Eclipse, CMS.