SUMMARY

• Over 8 years of administration experience including 3 years of experience with Hadoop Ecosystem in installation and configuration of different Hadoop eco - system components in the existing cluster.
• Experience in Hadoop Administration (HDFS, MAP REDUCE, HIVE, PIG, SQOOP, FLUME AND OOZIE, HBASE) NoSQL Administration
• Experience in deploying Hadoop cluster on Public and Private Cloud Environment like Amazon AWS, RackSpace and OpenStack.
• Setting up automated 24x7 monitoring and escalation infrastructure for Hadoop cluster using Nagios and Ganglia.
• Experience in installing Hadoop cluster using different distributions of Apache Hadoop, Cloudera and Hortonworks.
• Good Experience in understanding the client’s Big Data business requirements and transform it into Hadoop centric technologies.
• Analyzing the clients existing Hadoop infrastructure and understand the performance bottlenecks and provide the performance tuning accordingly.
• Installed, Configured and maintained HBASE
• Worked with Sqoop in Importing and exporting data from different databases like MySql, Oracle into HDFS and Hive.
• Defining job flows in Hadoop environment using tools like Oozie for data scrubbing and processing.
• Experience in configuring Zookeeper to provide Cluster coordination services.
• Loading logs from multiple sources directly into HDFS using tools like Flume.
• Good experience in performing minor and major upgrades.
• Experience in benchmarking, performing backup and recovery of Namenode metadata and data residing in the cluster.
• Familiar in commissioning and decommissioning of nodes on Hadoop Cluster.
• Adept at configuring NameNode High Availability.
• Worked on Disaster Management with Hadoop Cluster.
• Worked with Puppet for application deployment.
• Well experienced in building servers like DHCP, PXE with kick-start, DNS and NFS and used them in building infrastructure in a Linux Environment.
• Experienced in Linux Administration tasks like IP Management (IP Addressing, Subnetting, Ethernet Bonding and Static IP).
• Strong knowledge on Hadoop HDFS architecture and Map-Reduce framework.
• Experience in deploying and managing the multi-node development, testing and production
• Experience in understanding the security requirements for Hadoop and integrating with Kerberos authentication infrastructure- KDC server setup, creating realm /domain, managing
• Principles, generating key tab file for each and every service and managing key tab using key tab tools.
• Worked on setting up Name Node high availability for major production cluster and designed Automatic failover control using zookeeper and quorum journal nodes.
• Lead teams comprising of 2-3 members.

PROFESSIONAL EXPERIENCE

Confidential - Maryland

Hadoop Administrator

Responsibilities:

• Responsible for implementation and ongoing administration of Hadoop infrastructure.
• Monitored already configured cluster of 40 nodes.
• Installed and Configured cluster of 70 nodes
• Installed and configured Hadoop components Hdfs, Hive, HBase.
• Communicating with the development teams and attending daily meetings.
• Addressing and Troubleshooting issues on a daily basis.
• Launched R-statistical tool for statistical computing and Graphics.
• Working with data delivery teams to setup new Hadoop users. This job includes setting up Linux users, and testing HDFS, Hive.
• Supporting developers for deploying their jobs.
• Installed Redis and Configured HA for it. Previously they used RabbitMQ for mirroring the data but they upgraded to Redis.
• Confidential uses distribution IBM BigInsights
• Installed and Configured GDE for encrpting the data
• Upgraded BigInsights cluster from 2.1 to 4.0.
• Cluster maintenance as well as creation and removal of nodes.
• Monitor Hadoop cluster connectivity and security
• Manage and review Hadoop log files.
• File system management and monitoring.
• HDFS support and maintenance.
• Actively teaming with the infrastructure, network, database, application and business intelligence teams to guarantee high data quality and availability.
• Installed MicroStrategy on a Redhat Linux Environment using Postgresql for the Repository
• Deployed the MicroStartegyWeb on Tomcat in a Redhat linux Environment.
• Administered user and managed their access on Reports and Projects.
• Assisted in Evaluating the R Integrated pack and the D3 Visualization with MicroStrategy.
• Installed and configured Xming for Tomcat and Microstrategy.
• Knowledge on Scom, Splunk, SpringXD, Kafka.

Confidential - Richmond, VA

Hadoop Administrator

Responsibilities:

• Responsible for architecting Hadoop clusters Translation of functional and technical requirements into detailed architecture and design.
• Installed and configured multi-nodes fully distributed Hadoop cluster of large number of nodes.
• Provided Hadoop, OS, Hardware optimizations.
• Setting up the machines with Network Control, Static IP, Disabled Firewalls, Swap memory.
• Installed and configured Cloudera Manager for easy management of existing Hadoop cluster
• Administered and supported distribution of Hortonworks
• Worked on setting up high availability for major production cluster and designed automatic failover control using zookeeper and quorum journal nodes
• Implemented Fair scheduler on the job tracker to allocate fair amount of resources to small jobs.
• Performed operating system installation, Hadoop version updates using automation tools.
• Configured Oozie for workflow automation and coordination.
• Implemented rack aware topology on the Hadoop cluster.
• Importing and exporting structured data from different relational databases into HDFS and Hive using Sqoop
• Configured ZooKeeper to implement node coordination, in clustering support.
• Configured Flume for efficiently collecting, aggregating and moving large amounts of log data from many different sources to HDFS.
• Involved in collecting and aggregating large amounts of streaming data into HDFS using Flume and defined channel selectors to multiplex data into different sinks.
• Worked on developing scripts for performing benchmarking with Terasort/Teragen.
• Implemented Kerberos Security Authentication protocol for existing cluster.
• Good experience in troubleshoot production level issues in the cluster and its functionality.
• Backed up data on regular basis to a remote cluster using distcp.
• Regular Commissioning and Decommissioning of nodes depending upon the amount of data.
• Monitored and configured a test cluster on amazon web services for further testing process and gradual migration
• Installed and maintain puppet-based configuration management system
• Deployed Puppet, Puppet Dashboard, and PuppetDB for configuration management to existing infrastructure.
• Using Puppet configuration management to manage cluster.
• Experience working on API
• Generated reports using the Tableau report designer

Confidential, Nashville, TN

Hadoop Administrator

Responsibilities:

• Deployed a Hadoop cluster using cdh3 integrated with Nagios and Ganglia.
• Extensively involved in Cluster Capacity planning, Hardware planning, Installation, Performance Tuning of the Hadoop Cluster.
• Performed Installation and configuration of Hadoop Cluster of 90 Nodes with Cloudera distribution with CDH3.
• Installed Namenode, Secondary name node, Job Tracker, Data Node, Task tracker.
• Performed benchmarking and analysis using Test DFSIO and Terasort.
• Implemented Commissioning and Decommissioning of data nodes, killing the unresponsive task tracker and dealing with blacklisted task trackers.
• Implemented Rack Awareness for data locality optimization.
• Dumped the data from MYSQL database to HDFS and vice-versa using SQOOP.
• Used Ganglia and Nagios to monitor the cluster around the clock.
• Created a local YUM repository for installing and updating packages.
• Dumped the data from one cluster to other cluster by using DISTCP, and automated the dumping procedure using shell scripts.
• Implemented Name node backup using NFS.
• Performed various configurations, which includes, networking and IPTable, resolving hostnames, user accounts and file permissions, http, ftp, SSH keyless login.
• Worked with the Linux administration team to prepare and configure the systems to support Hadoop deployment
• Created volume groups, logical volumes and partitions on the Linux servers and mounted file systems on the created partitions.
• Implemented Capacity schedulers on the Job tracker to share the resources of the Cluster for the Map Reduce jobs given by the users.
• Worked on importing and exporting Data into HDFS and HIVE using Sqoop.
• Worked on analyzing Data with HIVE and PIG
• Helped in setting up Rack topology in the cluster.
• Helped in the day to day support for operation.
• Worked on performing minor upgrade from CDH3-u4 to CDH3-u6
• Upgraded the Hadoop cluster from cdh3 to cdh4.
• Implemented Fair scheduler on the job tracker to allocate the fair amount of resources to small jobs.
• Implemented Kerberos for authenticating all the services in Hadoop Cluster.
• Deployed Network file system for Name Node Metadata backup.
• Designed and allocated HDFS quotas for multiple groups.
• Configured and deployed hive metastore using MySQL and thrift server.

Confidential

Linux/MySQL Administrator

Responsibilities:

• Installation and configuration of Linux for new build environment.
• Created volume groups logical volumes and partitions on the Linux servers and mounted file systems and created partitions.
• Deep understanding of monitoring and troubleshooting mission critical Linux machines.
• Improve system performance by working with the development team to analyze, identify and resolve issues quickly.
• Ensured data recovery by implementing system and application level backups.
• Performed various configurations which include networking and IPTable, resolving host names and SSH keyless login.
• Managing Disk File Systems, Server Performance, Users Creation and Granting file access Permissions and RAID configurations.
• Automate administration tasks through the use of scripting and Job Scheduling using CRON.
• Installation and configuration of Linux for new build environment.
• Installing and maintaining the Linux servers
• Monitoring System Metrics and logs for any problems.
• Running cron-tab to back up data.
• Adding, removing, or updating user account information, resetting passwords, etc.
• Using Java Jdbc to load data into MySQL.
• Maintaining the MySQL server and Authentication to required users for databases.
• Creating and managing Logical volumes
• Installing and updating packages using YUM.
• Support pre-production and production support teams in the analysis of critical services and assists with maintenance operations.
• Automate administration tasks through use of scripting and Job Scheduling using CRON.

Confidential

MySQL Administrator

Responsibilities:

• Experience in MYSQL installation, configuration and migration
• Experience in managing Database Structure, Database Upgrades, Replication and Data Extraction
• Experience in Installation, configuration and maintenance of MSSQL Cluster.
• Define requirements for physical database consolidation, backup and recovery and application database disaster recovery.
• Installed and configured MySQL databases in different environments.
• Created Backup Scripts for MySQL databases.
• Prepared and configured Weekly and Daily Backup jobs.
• Restored MSSQL Backups and MySQL Backups to check backup integrity and practice disaster recovery.
• Set up and managed MySQL Master to Master replication, Master to Slave replication and MySQL Sharding.
• Troubleshot MySQL issues and optimized MySQL performance through slow query log and execution plan.
• Implemented MSSQL codes including procedures, functions, and triggers.
• Implemented MySQL procedures and ad hoc queries.
• Collaborated with system teams and development teams for codes release including MySQL database changes and patches.
• Upgraded MySQL from lower version 5.1 to higher version 5.5/5.6.