SUMMARY

• Worked on installation configuration and maintenance of 100+ node Hadoop cluster.
• Involved in collecting requirements from business users, designing and implementing data pipelines and ETL workflows end to end.
• Experience in performing various major and minor Hadoop upgraded on large environments.
• Experience in configuring, installing and managing Horton works, Cloudera & MapRHadoop Distributions
• Working experience on designing and implementing end - to-end Hadoop Infrastructure.
• Developed Hive Queries and automated those queries for analyzing on Hourly, Daily and Weekly basis
• Strong troubleshooting and performance tuning skills
• Coordinating Cluster services through Zookeeper
• Knowledge on ETL technologies, Machine Learning tools and Data mining.
• Experienced in managing Hadoop clusters using Cloudera Manager.
• Hands on experience in application development using Java, RDBMS, and Linux shell scripting
• Proficient in Core Java, JSP and Java Servlets.
• Contributed to Spring framework based application development.
• Worked on E-commerce, Travel and Medical domains.
• Worked on web designing using HTML, CSS, Java script, jQuery and Ajax.

TECHNICAL SKILLS

Hadoop eco system components: Hadoop, Mapreduce, yarn, hive, pig, sqoop, and flume.

Hadoop Clusters: Cloudera and Hortonworks

Tools: Tabula, toad,micro strategy integrations with hive.

Cloud Environments: Open stack, AWS, Rackspace.

Programming Languages: Unix Shell scripting, JAVA, SQL, C, C++

Monitoring and Alerting: Nagios, Ganglia

Operating Systems: Linux Centos 5,6, Red hat 6

PROFESSIONAL EXPERIENCE

Confidential, Bentonville

Hadoop Operations Engineer

Responsibilities:

• Installation, Configuration and Management of Hadoop Clusters using Cloudera Manager.
• Performed major and minor upgrades in large environments.
• 24x7 Monitoring and support through Nagios and Ganglia
• Managing the configuration of the clusters to meet the needs of analysis whether I/O bound or CPU bound.
• Experienced in managing and reviewing Hadoop Log files.
• Performing benchmark test on Hadoop clusters and tweak the solution based on test results.
• Supported users in running Pig and Hive queries and with the debugging.
• Responsible for troubleshooting issues in the execution of Map Reduce jobs by inspecting and reviewing log files.
• Performed Data scrubbing and processing with Oozie.
• Used Tableau to visualize the analyzed data.
• Involved in transferring data between RDBMS and HDFS using Sqoop.
• Installed and configured Kerberos for Hadoop and all of its eco system tools for security.
• Good understanding and related experience with Hadoop stack - internals, Hive, Pig and Map/Reduce
• Involved in managing and reviewing Hadoop log files
• Load and transform large sets of structured, semi structured and unstructured data
• Responsible to manage data coming from different sources
• Supported debugging of Map Reduce Programs of users.
• Involved in loading data from UNIX file system to HDFS.
• Involved in creating Hive tables, loading with data and writing hive queries.
• Implemented Partitioning, Dynamic Partitions, Buckets in HIVE.
• Monitor System health and logs and respond accordingly to any warning or failure conditions.
• Writing scripts to automate the data loading to cluster.
• Automated Cluster installations, Validations using scripts.
• Create, Execute and Debug SQL queries to perform data completeness, correctness, data transformation and data quality testing.

Confidential, New Jersey

Hadoop Administrator

Responsibilities:

• Involved in collecting requirements from business users, designing and implementing data pipelines and ETL workflows.
• Responsible for performance tuning, backup and recovery with respect to Hadoop systems.
• Involved in transferring data between RDBMS and HDFS using Sqoop.
• Responsible for troubleshooting issues in the execution of Map Reduce jobs by inspecting and reviewing log files.
• Created internal and external Hive tables and defined static and dynamic partitions as per requirement for optimized performance.
• Effectively used Oozie to develop automatic workflows of Sqoop, Map Reduce and Hive jobs.
• Used Tableau to visualize the analyzed data.
• Conducted root cause analysis and worked with users to troubleshoot map reduce job failures and issues with Hive and Map Reduce.

Confidential, CA

Hadoop/Linux Administrator

Responsibilities:

• Installed Hadoop on clustered environment
• Implemented DRBD for Name Node Metadata Backup
• Upgraded the cluster from CDHU1 to CDHU2 The tasks were first performed on the staging platform before doing it on production cluster
• Provided day to day production support of our Hadoop infrastructure including new hardware infrastructure and application installation
• Supported configuring, sizing, tuning and monitoring analytic clusters
• Monitoring cluster job performance and involved capacity planning
• Works with application teams to install operating system and Hadoop updates, patches, Version upgrades as required.
• Documented technical designs and procedures

Confidential

Software Engineer

Responsibilities:

• Planning and Development ofan e-commerce website using java/j2ee and spring framework.
• Experienced in using Spring(MVC Architecture, Security and Transaction Management) and Hibernate frameworks.
• Performing Unit Testing using JUNIT framework.
• Responsible for developing and maintaining all the entity and session beans.
• Experienced in database querying using MySQL.
• Proficient in using Jenkins for setting up the build periodically.
• Protecting user information and customer data using spring security.
• Worked on html, CSS, java script and jQuery for UI development.

Confidential

Java Developer

Responsibilities:

• Responsible for developing spring framework based applications.
• Experienced in using UI development technologies Html, CSS, Java Script and jQuery.
• Involved in analysis, design and development of the product.
• Responsible for development of configuration, mapping and Java beans for Persistent layer (Object and Relational Mapping) using Hibernate.
• Used Sub Version (SVN) for maintaining different versions and branches for the code.
• Support engineer for the clients using the product.
• Implemented various activities like transaction management and search operations that enable users to understand the product efficiently.
• Configured Web Logic resources such as JDBC providers, JDBC data sources, connection pooling and Java Mail sessions.
• Designed and developed the JSP pages using various J2EE technologies.
• Used log4j to log the messages for various database and spring framework related operations.