SUMMARY

• Over 5+ years of IT experience in Administering, Installing, Configuring and Maintaining Linux and Hadoop clusters and over 3 years of experience working on Hadoop eco - system.
• Experience in configuring, installing and managing MapR, Hortonworks & Cloudera Distributions.
• Experience in troubleshooting, finding root causes, Debugging and automating solutions for operational issues in the production environment.
• Experienced in Ansible YAML Scripts as the Software Configuration Management tool for automating Hadoop infrastructure changes.
• Involved in Hadoop cluster environment administration that includes adding and removing cluster nodes, performance tuning, cluster Monitoring, Troubleshooting.
• Expertise with Hadoop, MapReduce, Pig, Sqoop, Oozie and Hive.
• Good Understanding of the Hadoop Distributed File System and Eco System ( MapReduce , Pig , Hive , Sqoop and HBase )
• Experienced in analyzing Hadoop logs to troubleshoot production issues.
• Implemented MapR token based security for MapR clusters.
• Extensive knowledge on Hadoop HDFS architecture and MRv1, MRv2 (YARN) framework.
• Experienced in Commissioning and Decommissioning the Data nodes in Hadoop Cluster.
• Monitoring, Alerting and support through Ganglia and PepperData.
• Experienced in performing POCs to test the usability of new tools on top of Hadoop.
• Good working experience on Installing and maintaining the Linux servers.
• Knowledge of using RedHat Linux environment to manage internal systems and mange updates.
• Experience as an administrator of Windows Operating Systems ( XP, Vista and Windows 7,8 ) and Mac OS X
• Experience in setting up of Hadoop cluster in cloud services like AWS and Azure
• Hands-on experience in using windows Azure cloud computing platform.
• Worked with HBase and Cassandra NoSQL databases.
• Administering and maintaining Production, Stage, Development instances 24X7 Production Database Support and maintenance.
• Experience in Amazon AWS cloud services (EC2, S3)
• Experienced to schedule the jobs through Azkaban.
• Excellent project management skills and Excellent Communication & Analytical skills.

TECHNICAL SKILLS

Hadoop Ecosystems: HDFS, MapReduceV1, MapReduceV2 (YARN), Hive, Hue, Flume, Oozie, Zookeeper, Drill, Impala, HBase, Spark and Sqoop

Automation Tools: Ansible, Puppet, MCS, Ambari & Cloudera Manager

Hadoop Security: Kerberos, MapR security

Cloud Environments: AWS, Azure, Rackspace

Languages: C, SQL, Python, UNIX shell scripting

Monitoring and Alerting: Nagios, Ganglia

Database: MySQL

NoSQL Database: Hbase, Cassandra

Operating Systems: WinXP, Win 7 & 8, Linux (Ubuntu, CentOS, Red Hat), Mac OSX

PROFESSIONAL EXPERIENCE

Confidential, Boulder, CO

Hadoop Admin

Responsibilities:

• Experienced in Administration, Installing, Upgrading and Managing distributions of Hadoop clusters with MapR 5.1 on a cluster of 100+ nodes in different environments such as Development, Test and Production environments.
• Installed MapR clients and various Hadoop eco-systems using Ansible.
• Worked with cloud administrations like Amazon web services (AWS).
• Verify different scenarios in all the stages of the Data Lake after loading the data from source database into the respective tables using Azkaban scheduling tool
• Created workflows in Azkaban to automate the batch jobs
• Helped migrating and managing multiple applications from on premise to cloud using AWS services like S3, EC2.
• Experienced in storing the analyzed results into the Cassandra cluster.
• Explored Qubole Data Service platform for running AWS API, Hive Queries and Shell commands.
• Experienced in resolving the user tickets in day to day.
• Administering and maintaining Production, Stage, Development instances 24X7 Production Database Support and maintenance.
• Experience in managing the Hadoop MapR infrastructure with MCS.
• Working experience on maintaining MySQL databases creation and setting up the users and maintain the databases.

Confidential, San Jose, CA

Hadoop Administrator

Responsibilities:

• Maintaining the Operations, installations, configuration of 150+ node cluster with MapR distribution.
• Monitoring the health of the cluster and setting up alert scripts for memory usage on the edge nodes.
• Upgradation of ecosystem tools and deploying patches.
• Wrote shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
• Monitored disk memory, Heap, CPU utilization on all Master and Slave and took necessary measures to keep the cluster up and running on 24/7 basis.
• Configured Fair Scheduler to provide service-level agreements for multiple users of a cluster.
• Implemented MapR token based security.
• Worked on installing cluster, commissioning & decommissioning of data nodes.
• Setting up Yarn resources for user queues in fair share scheduler.
• Installed and configured Drill, Fuse and Impala on MapR-5.1.
• Created MapR DB tables and involved in loading data into those tables.
• Experience in analyzing data using Hive QL, Pig Latin, HBase and custom Map Reduce programs in Java.
• Hands on experience in analyzing Log files for Hadoop and eco system services and finding root cause.
• As an admin involved in Cluster maintenance, trouble shooting, Monitoring and followed proper backup& Recovery strategies.
• Involved in collecting metrics for Hadoop clusters using Ganglia, PepperData.
• Manage and automate systems configuration using Ansible.
• Installation and configuration of CLUSH utility for running the commands parallel across the cluster.

Confidential, NJ

Hadoop Administrator

Responsibilities:

• Experienced in Administration, Installing, Upgrading and Managing distributions of Hadoop clusters with Cloudera CDH4u5 using Cloudera Manager CM 4.6 on a cluster of 100+ nodes in different environments such as Development, Test and Production environments.
• Enabled HA for Name node using Cloudera Manager to avoid single point of failure
• Regular Maintenance of Commissioned/decommission nodes as disk failures occur using Cloudera Manager.
• Installation of hue for GUI access for hive, pig and Oozie.
• Installed Hadoop on clustered Environments on Dev/UAT/Prod Environments.
• Designed the rack topology for the production Hadoop cluster using CM.
• Develop benchmarking routines by using Teragen and Terasort.
• Implemented the Hadoop Name node HA services to make the Hadoop services highly available.
• Installed Ganglia to monitor Hadoop daemons and Implemented the changes in configuration parameters and in parallel monitored the changes in Ganglia.
• By using flume collected web logs from different sources and dumped them into HDFS.
• Implemented Oozie work-flow for ETL Process.
• Exporting data from RDBMS to HIVE, HDFS and HIVE, HDFS to RDBMS by using SQOOP.
• Implemented shell scripts for log-Rolling day to day processes and made it automated.
• Deployed cron job for backing up the Name node metadata.
• Worked on file system management and monitoring and Capacity planning.
• Execute system disaster recovery processes.
• Work with the project and application development teams to implement new business initiatives as they relate to Hadoop.
• Expertise in managing the Hadoop infrastructure with Cloudera Manager.
• Working with data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos principals and testing HDFS, Hive, Pig and MapReduce access for the new users.
• Demonstrate and understanding of concepts, best practices and functions to implement a Big Data solution in a corporate environment.
• Configuring Kerberos and AD/LDAP for Hadoop cluster
• Responsible in creating Shell scripts and Python scripts for back up of metadata for all Hadoop components and backing up configuration xml files.
• Experience in administering the Linux systems to deploy Hadoop cluster and monitoring the cluster using Ganglia
• Demonstrate and understanding of concepts, best practices and functions to implement a Big Data solution in a corporate environment.
• Installed various Hadoop eco-systems using through Ansible.
• Experience in using Source control repositories like GIT .

Confidential, CA

Hadoop Admin

Responsibilities:

• Managing Hadoop clusters Prod, Staging, Qa and Dev .
• Deployed 100+ nodes Hortonworks Hadoop Cluster HDP 2.1 using Ambari 1.5.1.
• Monitoring and support through Nagios.
• Worked with Hortonworks support to resolve the issues.
• Experienced in managing and reviewing Hadoop Log files.
• Involved in Hadoop cluster environment administration that includes De-commissioning and commissioning nodes, balancing and cluster capacity planning.
• Worked on High Availability for Name Node using Ambari to avoid single point of failure.
• Build and manage development and testing environments, assisting administrators in debugging application issues.
• Worked on Creating Kafka topics, partitions, and writing custom practitioner classes.
• Hands on experience on automation at work using Puppet.
• Migrated the data from production cluster to DR cluster.
• Integrating and Kerberos Hadoop cluster with LDAP and active directory for security.
• Configured Fair Scheduler to provide service-level agreements for multiple users of a cluster
• Benchmarking Hadoop cluster and tuning Hadoop configurations.
• Defining and documenting operational processes and monitoring and maintenance procedures
• Experience working with Hadoop Ecosystem ( Hive, Pig, Oozie, Sqoop ).
• Developing scripts for monitoring the cluster and data loading.
• Supporting users on day-to-day tickets and issues.
• Contribute to the creation and maintenance of system documentation.
• Working experience in supporting and deploying in an AWS environment including the following services: EC2, S3 and EMR.
• Deploying EMR cluster for users on demand in amazon elastic MapReduce.