SUMMARY

• 9 years of professional experience including 3 years of experience in Hadoop administration.
• As a Hadoop administration responsibilities include software installation, configuration, software updates, backup and recovery, commissioning and decommissioning data nodes, cluster setup cluster performance and monitoring on daily basis, maintaining cluster on healthy on different Hadoop distributions (Hortonworks and Cloudera).
• Experience in installation, management and monitoring of Hadoop cluster using pivotal command center, Cloudera Manger andAmbari.
• Strong experience in configuring Hadoop ecosystem tools with including Pig, Hive, Hbase, Sqoop, Flume, Kafka, Spark, Oozie, and Zookeeper.
• Installed and configured HDFS (Hadoop Distributed File System), MapReduce and developed multiple MapReduce jobs for data cleaning.
• Strong understanding on Hadoop architecture and MapReduce framework.
• Experience in deploying Hadoop 2.x (YARN).
• Optimized the configurations of Map Reduce, Pig and Hive jobs for better performance.
• Worked on Building real time pipeline for streaming data using Kafka Streaming.
• Expertise usingApache Sparkfast engine for large - scale data processing.
• Experience in transferring data between HDFS and Relational Database with Sqoop.
• Experience in configuring Hadoop based monitoring tools- Nagios, Ganglia
• Involved in cluster maintenance, bug fixing, and troubleshooting monitoring and followed proper backup and recovery strategies.
• Experience on commissioning, decommissioning, balancing and managing nodes and tuning server for optimal performance of the cluster.
• Strong knowledge in Hadoop cluster capacity planning, performance tuning, and cluster monitoring troubleshooting.
• Strong knowledge on setting up automatic failover control and manual failover control using Zookeeper and quorum journal nodes.
• Implement and manage Secure Authentication mechanism for Hadoop clusters using Kerberos and IPTABLE rules.
• Good Experience in setting up the Linux environments, Password lessSSH, Creating file systems, disabling firewalls, swappiness, Selinux and installing Java.
• Managed various environments like CentOS, Redhatlinuxand Windows server2008/2012.
• Hands on experience on cluster upgradation and patch upgrade without any data loss and with proper backup plans.
• Superior skills in communication, strong initiative for learning new skills and conquering challenges.

TECHNICAL SKILLS

Hadoop ECOSYSTEM: HDFS, MapReduce, Hive, Pig, Sqoop, Oozie, Flume, Spark, Zookeeperand Kafka.

SCripting: Shell Scripting

CLUSTER MANAGEMENT TOOL: Pivotal, Ambari, Cloudera

MONITORING TOOLS: Nagios, Ganglia

DATABASE: MySQL, MS SQL Server 2008/2012

SERVERS: Apache Tomcat Server, Apache HTTP Web Server

Operating system: Mac, Linux, Windows Servers 2008/2012

PROFESSIONAL EXPERIENCE

Confidential, Newark, CA

Hadoop Administrator

Responsibilities:

• Involved in installing, configuring and using Hadoop Ecosystems (Hortonworks, pivotal command Centre).
• Day to day responsibilities includes solving developer issues, providing access to new users and providing instant solutions to reduce the impact and documenting the same and preventing future issues.
• Responsible in administrating and maintaining Hadoop cluster PHD v2.1
• Experience Monitoring & tuning HDFS for optimal performance and uptime.
• Experience on configuration of Kafka.
• Built real time pipeline for streaming data using Kafka Streaming.
• Expertise usingApache Sparkfast engine for large-scale data processing
• Experienced in managing and reviewingHadooplog files.
• Involved in cluster maintenance, bug fixing, and troubleshooting monitoring and followed proper backup and recovery strategies.
• Implemented open source monitoring tool Ganglia for monitoring the various services across the cluster.
• Supported Map Reduce Programs those are running on the cluster.
• Experience in Hadoop cluster tasks like Adding and Removing Nodes without any effect to running jobs and data.
• Working with data delivery teams to setup new Hadoop users and environment. This job includes setting up Linux users, setting up Kerberos principals and testing HDFS, Hive, Pig and MapReduce/YARN access for the new users.
• Involved in development/implementation of redhatHadoopenvironment.
• Involved in creating Hive tables, loading with data and writing hive queries, which will run internally in map.
• Worked with the applications team to install the operating systems, Hadoop updates, patches and version upgrades as required.

Confidential

Hadoop Administrator

Responsibilities:

• Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting,data backups, Manage& review log files.
• Experienced on installation of new components and removal of them through Ambari.
• Monitoring systems and services through Ambari dashboard to make the clusters available for the business.
• Changing the configurations based on the requirements of the users for the better performance of the jobs.
• Experienced inAmbari-alerts configuration for various components and managing the alerts.
• Strong Experience in Installation and configuration of Hadoop ecosystem like Yarn, HBase, Flume, Hive, Pig, Sqoop.
• Expertise in Hadoop cluster task like Adding and Removing Nodes without any effect to running jobs and data.
• Load log data into HDFS using Flume.
• Worked extensively in creating MapReduce jobs to power data for search and aggregation.
• Worked extensively withSqoop for importing data.
• Extensively used Pig for data cleansing.
• Scheduled Oozie workflow engine to run multiple Hive and Pig jobs, which independently run with time and data availability.
• Worked on pulling the data from relational databases, Hive into the Hadoop cluster using the Sqoop import for visualization and analysis.
• Hand on experience on cluster upgradation and patch upgrade without any data loss and with proper backup plans.

Confidential

Hadoop & Linux Administrator

Responsibilities:

• Configuring Hadoop Eco-system
• Performance Optimization of Hadoop cluster based on job requirements
• Monitor a Hadoop cluster and execute routine administration procedures
• Dumping the database data into HDFS by using Sqoop.
• Analyzing the customer data for identifying the long term customers and fraud customers by using Hive
• Installing and maintaining the Linux servers.
• Created volume groups logical volumes and partitions on the Linux servers and mounted file systems and created partitions.
• Creation, Installation and administration of Red Hat Virtual machines in VMware Environment.
• Deep understanding of monitoring and troubleshooting mission critical Linux machines.
• Installed Cent OS using Pre-Execution environment boot and Kick start method on multiple servers.
• Running Cron-tab to back up data.
• Adding, removing, or updating user account information, resetting passwords, etc.
• Performance tuning of Virtual Memory, CPU, system usage in Linux and Solaris servers.
• Supporting infrastructure environment comprising of RHEL and Solaris and AIX.
• Involved in development/implementation of CentosHadoopenvironment.

Confidential

Linux Administrator

Responsibilities:

• Building and supporting environments consisting Testing, Contingency, Production and Disaster Recovery servers.
• Implemented Jumpstart on Solaris and Kick start for Redhat environments.
• Performed patching of RHEL using yum, up2date package management system utilities for effective package maintenance.
• Supporting infrastructure environment comprising of RHEL and Solaris and AIX.
• Experience with VMware Virtualization using ESX hypervisor of VSphere.
• Create, extend, reduce and administration of Logical Volume Manager (LVM) in RHEL environment.
• Creation, Installation and administration of Red Hat Virtual machines in VMware Environment.
• Rebuilding of kernel in RHEL using mkinitrd image and monitored logs in in-built directories during server bootup.
• Worked on projects like PCI, SR to ensure all goes well and provided support till servers go in to production environment.
• Configuration of Network bonding which include Active/Standby and Active/Active.
• Troubleshooting Network, memory, CPU, swap and File system issues, TCP/IP, NFS, DNS, SMTP in Linux and Solaris servers.
• Performance tuning of Virtual Memory, CPU, system usage in Linux and Solaris servers.
• Supported class monitoring and management tools such as Open NMS, Tivoli and VCO.
• Performance Monitoring and Performance Tuning using Top, prstat, SAR, vmstat, ps, iostat.
• Package management using RPM, YUM and UP2DATE in RHEL
• Performed Disaster Recovery in RHEL servers which consists of LVM based FS and Red Hat Clustering.
• Installation, configuration and administration of Jboss, Apache, Tomcat and Web Sphere.
• Develop, Maintain, update various script for services (start, stop, restart, recycle, cron jobs) Unix based Korn shell, Bash.
• Schedule jobs using Crontabs on Linux servers
• User, Group, Package administration, various repetitive activities across Linux Environment.
• Creation, initialization, addition of Oracle ASM and FSR devices in multipath environment
• Creation of Jumpstart and Kickstart configuration for the automatic provision of servers using Bladelogic.