SUMMARY:

• 8 years of professional IT experience which includes about 4 years of proven experience in Hadoop Administration in deploying, maintaining, monitoring and upgrading Hadoop Clusters using Apache, Cloudera (CDH), Hortonworks (HDP) Distributions.
• Hands on experience on configuring a Hadoop cluster for a production environment, Amazon Web Services (AWS) using an EC2 instance.
• Experience in using Ambari, Cloudera Manager for installation and management of Hadoop Cluster.
• Excellent understanding ofHadoopEcosystem including HDFS, MapReduce, Hive, Pig, Kafka, YARN, HBase, Oozie, ZooKeeper, Flume and Sqoop based Big Data Platforms.
• Strong Knowledge in configuring NameNode High Availability.
• Experience in managing Hadoop infrastructure like commissioning, log rotation, and rack topology implementations.
• Experience working with, extending, and enhancing monitoring systems like Nagios.
• Experience with setting up and deploying NoSQL database like HBase.
• Experience in dealing with structured, semi - structured and unstructured data in Hadoop eco system.
• Experience in a variety of Relational Databases like MySQL, MSSQL, and Oracle.
• Expertise in Hive Query Language (HiveQL), Hive Security and debugging Hive issues.
• Experience in using Chefas the Software Configuration Management tool for automating Hadoop infrastructure changes.
• Adept with Linux environment setup, maintenance and troubleshooting
• Experience in upgrading the existingHadoopcluster to latest releases.
• Experience in configuring Zookeeper to provide Cluster coordination services.
• Experience in developing Shell Scripts for system management.
• Experience in providing security forHadoopCluster with Kerberos.
• Working experience with Jira and fast-paced Agile / Scrum environment.
• Effective problem solving skills and outstanding interpersonal skills. Ability to work independently as well as within a team environment. Driven to meet deadlines.
• Motivated to produce robust, high-performance software.
• Ability to learn and use new technologies quickly.

TECHNICAL SKILLS:

Methodologies: Agile, Rapid Application Development, Waterfall Model, And Iterative Model

Operating Systems: Windows 98/NT/XP/Vista/7, Linux (Ubuntu, CentOS, RedHat), MacOSX.

Hadoop Platforms: Hortonworks, Cloudera, Amazon AWS, Apache

Hadoop Ecosystem: HDFS, MapReduce, Pig, Hive, Sqoop, Oozie, Flume, Storm, Knox, HBase, Ambari, CDH, HDP

Languages: Java, C, Pig Latin, PHP.

Databases (RDBMS and NoSQL): MySQL, Oracle, Hbase, DB2.

Development Tools & IDEs: Maven, Composer, Git, Eclipse, Sublime Text, NetBeans

Web Technologies: JSP, REST API, HTML5, CSS, JavaScript, JQuery

JEE Technologies Servlets, Web Services,: Apache Tomcat

Frameworks: Hadoop, Laravel.

PROFESSIONAL EXPERIENCE

Confidential

Sr.Hadoop Administrator

Responsibilities:

• Responsible for Administration, installing, managing and upgrading distributions of Hadoopwith Apache & Hortonworks cluster in different environments such as development, stage and production (Operational & Analytics) environments.
• Responsible for cluster maintenance, commissioning and decommissioning of nodes cluster monitoring and troubleshooting.
• Performed ambari and HDP upgrades
• Install and setup Azkaban for developers to schedule jobs.
• Implemented Capacity Schedulers to share the resources of the cluster for the MapReduce jobs given by the users
• Set up Kafka for the proof of concept for carrying out log processing
• Worked with Hortonworks support to resolve issues with services upgrade
• Implement HiveServer and Metastore HA and migrated jobs to use zookeeper discovery
• Enabled PAM authentication for Hive and migrated all jobs without downtime
• Performed hive tuning for faster results and efficient usage of resources
• Work with users (analysts and developers) to resolve issues related to jobs
• Work with Linux team to carry out hardware maintenance
• Set up elastic pipelines using beats, logstash, Kafka and elastic
• Created dashboards using kibana
• Installed, maintained and upgraded elasticsearch
• Worked on setting up high availability for production cluster and designed automatic failover control using Zookeeper.
• Responsible for managing file system check for blocks & directories onHadoopcluster to ensure no block are missing.
• Monitored and analyzed the map reduces job executions on the cluster.
• Analyzed the inputs to the development regarding the efficient utilization of resources like memory and CPU utilization.
• Handled the upgrades and patch updates.
• Monitored and controlled local file system disk space usage, local log files, cleaning log files with automated scripts.
• Worked withHadoopdevelopers, designers in troubleshooting map reduce job failures and issues and helping to developers.
• Provide in-depth technical and business knowledge to ensure efficient design, programming, implementation and on-going support for the application.

Confidential, Cincinnati, Ohio

Sr.Hadoop Administrator

Responsibilities:

• Responsiblefor Administration, installing, managing and upgrading distributions of Hadoopwith Apache & Hortonworks (HDP)on aCluster of 400+ Nodes in different environments such as Development, Test and Production (Operational & Analytics) environments.
• Worked with network and Linux system engineers/admin to define optimum network configurations, server hardware and operating system.
• Administer and support distribution of Hortonworks.
• Assisted in design of scalable Big Data clusters and solutions.
• Design and Configure the Cluster with the services required (Hive server2, Kerberos, HDFS, Hue, Hive, Hbase, Zookeeper).
• Deployed high availability on the Hadoop cluster using quorum journal nodes.
• Troubleshoot the Hadoop cluster by reviewing Hadoop service log files to find the root cause.
• Implemented automatic failover using zookeeper and zookeeper failover controller.
• Implemented Rack Awareness for data locality optimization.
• Involved in collecting and aggregating large amounts of streaming data and every node logs into HDFS using Flume.
• Monitoring and controlling local file system disk space usage, log files, cleaning log files with automated scripts.
• Implemented Capacity Scheduler to share the resources of the cluster for the MapReduce jobs given by the users.
• Used Chef to manage OS dependencies and deploy third party packages.
• Experience in setting up Sqoop workflows to migrate data to and fro from HDFS and RDBMS (MySQL, Teradata) to assist further OLAP analytics on the retrieved data via Hive.
• Experience in setting up perimeter level security forall components of Hadoop Cluster using Kerberos with ADintegration.
• Worked with Kafka for the proof of concept for carrying out log processing on distributed system.
• Installed and configured Spark ecosystem components (SparkSQL, SparkStreaming, MLlib)
• Involved in defining job flows using Oozie for scheduling jobs.
• Conducting root cause analysis and resolve production problems and data issues.

Confidential, Houston, TX

Hadoop Administrator

Responsibilities:

• Support and enhance the capabilities of an existing HDP cluster by adding nodes.
• Worked with Unix and Networking team for racking and provisioning of bare metal servers.
• Deployed Hortonworks Hadoop Ecosystem components such as Sqoop, Hbase and Mapreduce. Deployed 300+ node Hortonworks Hadoop Cluster (HDP 2.1) using Ambari 1.5.1.
• Involved in Hadoop cluster environment administration that includes De-commissioning and commissioning nodes, balancing and cluster capacity planning.
• Worked on High Availability for Name Node to avoid single point of failure.
• Enabled High Availability for Resource Manager, Oozie, Hive Server 2, Hive Metastore on production clusters with minimal to zero impact on existing jobs and scheduled workflows
• Participated in development and execution of system and disaster recovery process.
• Performed an Ambari upgrades and HDP Stack upgrades in production environment from HDP 1.3 through HDP 2.1
• Monitor workload, job performance, and capacity planning.
• Monitor Hadoop cluster environments through Nagios.
• Build and manage development and testing environments, assisting developers in debugging application issues.
• Deployed and supported Hortonworks Hadoop cluster in AWS EC2 environment.
• Configured Fair Scheduler to provide service-level agreements for various teams.
• Enabled job queues and integrated it with Active Directory to control access on per user basis.
• Working with Hortonworks support to resolve the issues.
• Integrated Hadoop cluster with LDAP and active directory for security.
• Implemented and automated HDFS snapshots for critical data.
• Defining and documenting operational processes and monitoring and maintenance procedures
• Monitoring the cluster on a daily basis, check the error logs and debug issues.

Hadoop Administrator

Responsibilities:

• Installed, Configured and maintained Apache Hadoop clusters for application development.
• Worked on setting up high availability for production cluster and designed automatic failover control using Zookeeper.
• Developed shell scripts to monitor health check up of Hadoop daemon services and respond accordingly.
• Implemented rack aware topology on Hadoop Cluster.
• Involved in database performance tuning and data modeling.
• Responsible for managing file system check for blocks & directories onHadoopcluster to ensure no block are missing.
• Responsible for cluster maintenance, commissioning and decommissioning of nodes cluster monitoring and troubleshooting.
• Monitored and analyzed the map reduce job executions on the cluster.
• Analyzed the inputs to the development regarding the efficient utilization of resources like memory and CPU utilization.
• Handled the upgrades and patch updates.
• Involved in loading data from UNIX file system to HDFS.
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
• Monitored and controlled local file system disk space usage, local log files, cleaning log files with automated scripts.
• Involved in creating HIVE tables, loading with data and writing HIVE queries.
• Worked withHadoopdevelopers, designers in troubleshooting map reduce job failures and issues and helping to developers.
• Provide in-depth technical and business knowledge to ensure efficient design, programming, implementation and on-going support for the application.

Confidential

Linux System Administrator

Responsibilities:

• Administration of RHEL, which includes installation, testing, tuning, upgrading and loading patches, troubleshooting server issues.
• Coordinated with technical teams for installation of Hadoop and third party related applications.
• Developed strategies for regular operations in order to improve availability and proper utilization.
• Involved in troubleshooting problems and issues related to the efficient, secure operations of the Linux operating system.
• Managed patching, monitoring system performance and network communication, backups, risk mitigation, troubleshooting, software upgrades and modifications of the Linux servers
• Perform backup, file replications and script management servers.
• Resolved DNS related issues.
• Troubleshooting PC hardware and software.
• Enabled remote access by constructing and managing Cisco VPN client.
• Solving network issues with network applications.
• Used Performance Scripts to troubleshoot the issues that arose during the backend disk failures.
• Good Understanding of TCP/IP networking and its security considerations.
• Responsible for scheduling the Cron Jobs.
• Analyze the Performance of the Linux System to identify Memory, disk I/O and network problem.
• Responsible for logs and Resource Monitoring.
• Responsible for patch installations and updates on the servers

Confidential

Linux System Administrator

Responsibilities:

• Provide system administration services like backup, monitoring, installation, configuration and user permissions.
• Installed Cent OS using Pre-Execution environment boot and Kick-start automation method on multiple servers.
• Schedule backup jobs and object level recovery using maxima tool.
• Dealing with hardware failures by working with vendors.
• Keeping systems up to date with latest stable releases of OS.
• Creating new users, Resetting user passwords, Lock/Unlock user accounts
• Installed and maintained software packages (.rpm) as required.
• Monitoring System Metrics and logs for any problems.
• Running crontab to back up data.
• Involved in Adding, removing, or updatinguser accountinformation, resettingpasswords etc.
• Took Backup at regular intervals and planned with a good disaster recovery plan.
• Documented and maintain server, network, and support documentation including application diagrams.