We provide IT Staff Augmentation Services!

Hadoop Administrator Resume

San Antonio, TX

SUMMARY:

  • Around 8 years of experience in Information Technology along with 3+ year of experience in Cloudera and Horton works as Hadoop Administrator .
  • In - depth understanding/knowledge of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce concepts.
  • Experience in installation and configuration of Hadoop ecosystems like HDFS , Hive , Yarn , HBase , Sqoop, Flume , Oozie , Pig , and Spark .
  • Experienced in installation, configuration, supporting and monitoring 100+ node Hadoop cluster using Cloudera manager and Hortonworks distributions.
  • Experience in Adding and removing the nodes in Hadoop Cluster.
  • As an admin involved in Cluster maintenance, troubleshooting, Monitoring and followed proper backup& Recovery strategies.
  • Hands-on experience in configuration and management of security for Hadoop cluster using Kerberos .
  • Moving the data from Teradata into HDFS using Sqoop and importing various formats of flat files into HDFS.
  • Experience in benchmarking, performing backup and disaster recovery of Name Node metadata and important sensitive data residing on thecluster.
  • Experience in setting up and managing the High-Availability to avoid asingle point of failure on large Hadoop Clusters.
  • Knowledgeable of spark and Scala mainly in framework exploration for thetransition from Hadoop /Map Reduce to spark .
  • Working with applications teams to install theoperating system, Hadoop updates, patches, version upgrades as required.
  • Good Knowledge in Amazon AWS concepts like EMR, S3, and EC2 web services which provide fast and efficient processing of Hadoop .
  • Experience in writing Shell scripts for various purposes like file validation, automation and job scheduling using Crontab.
  • Experience in developing Shell Scripts for system management.
  • Knowledge on Vmware esx, Esxi, Vcenter, Splunk.
  • Experience in installing and monitoring the Hadoop cluster resources using Nagios.
  • Strong communication skills with a professional attitude and can take the pressures to drive with enthusiasm to support client Hadoop cluster with full potential and learn new technologies.

PROFESSIONAL EXPERIENCE:

Confidential, San Antonio,TX

Hadoop Administrator

Responsibilities:

  • Responsible for cluster maintenance, monitoring, commissioning and decommissioning Data nodes, Troubleshooting, manage and review backups, manage and review log files.
  • Involved in cluster capacity planning, Hardware planning, Installation, Performance Tuning of the Hadoop cluster.
  • Worked with the technical architect in upgrading and increasing the size of Hadoop cluster.
  • Coordinate with different teams with user issues and resolved it.
  • Monitored the Hadoop cluster with Ambari GUI to ensure the health of Hadoop services in Hadoop cluster
  • Connected with Hortonworks support team for resolving issues as well as preferred recommendations.
  • Day to day responsibilities includes solving developer issues, deployments, moving code from one environment to another environment, providing access to the new user, providing instant solutions for reducing the impact and documenting the same and preventing future issues.
  • Planned and prepared the use case for new Hadoop services and tested on sandbox by adding/installing using Ambari manager.
  • Working experience in designing and implementing complete end-to-end Hadoop Infrastructure which includes all Hadoop Ecosystem.
  • Upgraded HDP from 2.2 to HDP 2.4.2.
  • Experience in importing and exporting terabytes of data using Sqoop from Relational Database Systems to HDFS.
  • Experience in providing support to data analyst in running Pig and Hive queries.
  • Configured Capacity scheduler with various queues and priority for Hadoop
  • Set up and manage High Availability Namenode, Resource Manager, Hive Metastore, and Oozie to avoid a single point of failures in large clusters.
  • Designed and allocated HDFS quotas for multiple groups.
  • Created HIVE databases and granted appropriate permissions through Ranger policies.
  • Introduced SmartSense and got optimal recommendations from the vendor and even for troubleshooting the issues.
  • Moving the data from Teradata into HDFS using Sqoop and importing various formats of flat files into HDFS.
  • Used Oozie scheduler to automate the pipeline workflow and orchestrate the sqoop, hive and pig jobs that extract the data on a timely manner.
  • Written complex Hive and SQL queries for data analysis to meet business requirements.
  • Exported analyzed data to downstream systems using Sqoop for generating end-user reports, Business Analysis reports and payment reports.
  • Development operations using GIT, Puppet, its modules configuration, upload to master server and implement on client servers.

Environment: HDFS, Map Reduce, Hortonworks, Hive, Pig, Flume, Oozie, Sqoop, Ambari, and Linux.

Confidential, Hillard, OH

Hadoop Administrator

Responsibilities:

  • Installed Hadoop ecosystem components like Pig, Hive, HBase and Sqoop in a Cluster.
  • Adding/Installation of new components and removal of them through Cloudera.
  • Monitoring workload, job performance, capacity planning using Cloudera.
  • Major and Minor upgrades and patch updates.
  • Creating and managing the Cron jobs.
  • Experience in setting up tools like Nagios for monitoring Hadoop cluster.
  • Handling the data movement between HDFS and different web sources using Flume and Sqoop.
  • Extracted files from NoSQL database like HBase through Sqoop and placed in HDFS for processing.
  • Installed Oozie workflow engine to run multiple Hive and Pig jobs.
  • Building and maintaining scalable data pipelines using the Hadoop ecosystem and other open sources components like Hive and HBase.
  • Installed and configured HA of Hue to point Hadoop Cluster in Cloudera Manager.
  • Working with applications teams to install theoperating system, Hadoop updates, patches, version upgrades as required.
  • Responsible for developing data pipeline using HDInsight, Flume, Sqoop and Pig to extract the data from weblogs and store in HDFS.
  • Performed transformations, cleaning and filtering on imported data using Hive, Map Reduce, and loaded final data into HDFS.
  • Commissioning Data Nodes when data grew and De-commissioning of data nodes from thecluster in hardware degraded.
  • Set up and managing HA Name Node to avoid asingle point of failures in large clusters.
  • Working with data delivery teams to setup new Hadoop users, Linux users, setting up Kerberos principles and testing HDFS, Hive.
  • Discussions with other technical teams on regular basis regarding upgrades, process changes, any special processing, and feedback.

Environment: Linux, Shell Scripting, Map Reduce, SQL server, NoSQL, Cloudera, Flume, Sqoop, Pig, Hive, Zookeeper, and HBase.

Confidential

Linux System Administrator

Responsibilities:

  • Managing all types of installation and upgrade on Linux servers.
  • Customized User environments, Administered User Accounts on Red Hat Linux servers.
  • Created and managed user/group accounts, password and setting up permissions.
  • Monitoring backup using Backup Exac, regularly monitored Alert-log Files and trace files on the day-to-day basis.
  • Planning, installation, configuration, management and troubleshooting of Red Hat Enterprise Linux.
  • Creation and management of approved user accounts and groups, and created and managed documentation on the configuration and support of the Oracle databases
  • Manage file system utilization using script scheduled as a cron job
  • Performed automation with simple shell scripting
  • Maintain file system and host security using chmod, chown, chgrp.
  • Perform software package installation, upgrades, and updates using yum and RPM commands and manage filesystems using fdisk and LVM.

Environment: Red Hat Linux, Shell Scripting, Oracle.

Confidential

Linux System Administrator

Responsibilities:

  • Installation of patches and packages using RPM and yum in Red hat Linux.
  • Created and modified application related objects, created Profiles, users, roles and maintained system security.
  • Responsible for setting up cron jobs scripts on production servers.
  • Monitored System Activities like CPU, Memory, Disk and Swap space usage to avoid any performance issues.
  • Worked on reliability, manage liability and performance improvements for Linux enterprises.
  • Responsible for writing/modifying scripts using bash for day-to-day administration.
  • Modified Kernel parameters to improve the server performance in Linux.
  • Creation of Logical volumes (LVM) for Linux operating systems.
  • Involved in design, configuration, installation, implementation, management, maintain and support for the Corporate Linux servers RHEL 4x, 5.x, CENTOS 5.x.
  • Maintained proper documentation of all the activities carried out during the project.
  • Worked with DBA team for database performance issues, network related issues on Linux Servers.
  • Created and modified users and groups with SUDO permission.
  • Coordinating users for any server activities which may involve major changes on software or any hardware related issues.
  • Performing Backups and restores as required by the SLAs of the organization.

Environment: Red Hat Linux, Logical Volume Manager, Global File System, Red Hat Cluster Servers, Oracle.

Hire Now