Hadoop Administrator Resume
PA
SUMMARY:
- 6.5 years of experience in IT Infrastructure Administration with last 2.5 years of experience in administration, implementation & management of Big Data Hadoop Clusters using Cloudera distribution.
- Hands on experience in administration, installing, configuring and troubleshooting of Apache Hadoop ecosystem components like HDFS, Yarn, MapReduce1, Pig, Hive, Impala, Sqoop, Hbase (NoSql), Zookeeper, Flume, Kafka, Solr, Sentry.
- Experience in managing and installing scalable and secure Hadoop (CDH) clusters including role base authorization using Sentry as well as authentication using Kerberos.
- Experience in configuring HDFS High Availability (HA) on the production cluster with the deployment of High Availability of NameNode on the cluster.
- Experience in Backup and recovery of Cloudera Manager Servers.
- Having excellent knowledge in data processing and streaming using Spark framework.
- Experience in administration of Kafka and Flume streaming using Cloudera Distribution.
- Hands on experience on Impala Cluster used to process large set of data.
- Extensive experience in performing system administration for Red Hat Enterprise Linux 5x/6x/7x and VMware ESXi 5.4/5.5 and Windows 2003/2008/2012 R2 Server with Hyper - V.
- Provision, monitor and maintain AWS EC2 instances, watching the security and manage the AWS S3 bucket storage on AWS cloud environment.
- Core and In-depth knowledge of AWS services such as EC2, Auto Scaling, ELB, EBS, S3, RDS and IAM.
- Having good understanding of Hortonworks (HDP) and Ambari tool.
- Experience in large scale data migration of LINUX Logical Volumes (LVM) and restore the data using AWS snapshots.
- Hands on experience of installation, configuration, support and maintenance of various enterprises Red Hat Enterprise LINUX (RHEL) Servers.
- Extensive experience in administration, maintenance and troubleshooting which includes provisioning and decommissioning of Storage Arrays like IBM System Storage DS 8300/8700/8800 & Storwize V7000 and EMC Storage Symmetrix VMAX and VNX.
- Hands on experience in Centrify Server Suite to perform identity, authentication and access management for Linux servers within Windows Active Directory.
- Experience in managing enterprise level Backup solutions using CommVault v10/11 and IBM TSM 5.5/6 (Tivoli Storage Manager) backup environment, also administration and management of IBM TS3500 Tape Libraries across two sites (DC and DR).
- Hands on experience in CommVault version 10 & 11 Backup Administration.
- Perform backup and restoration procedures, support audit efforts and ensure compliance with corporate and regulatory standards (i.e. NERC CIP & SOX) standards and provide 24x7 technical supports on a rotational basis.
- Having experience with North American Electric Reliability Corporation, Critical Infrastructure Protection (NERC CIP & SOX) regulatory standards and requirements.
- Experience in SolarWinds and Nagios for enterprise server and application monitoring, storage resource monitoring, Network performance and DNS monitoring.
- Hands on experience on ITIL based IT Service Management (ITSM) solution tool ServiceNow.
TECHNICAL SKILLS:
Operating Systems: Red Hat Enterprise Linux (RHEL 5.x/6.x/7.x), Windows Server 2003/08/12R2, VMware EXSi 5.x, CentOS, AIX, Oracle VM (OVM)
Big Data Hadoop: HDFS, Hive, Hue, HBase (NoSql), MapReduce v1, Yarn, Zookeeper, Pig, Sqoop, Impala, Flume, Spark, Kafka, Kerberos, Sentry, Solr
Hadoop Distribution: Cloudera, Hortonworks
Servers Hardware: HPE ProLiant DL380 Gen9,IBM x3650, x3750, x3850-m2, x3650-M2, IBM Power p7 series
Storage Hardware: EMC VNX 5300, Symmetrix DMX-2000/3000 & VMAX 10k/20K, IBM DS 8700/8800/8300, IBM Storwize V7000
AWS: EC2, Auto Scaling, ELB, EBS, S3, IAM, CloudWatch, RDS, VPC, Route 53
Backup: Commvault 10 & 11, IBM TS3500 Tape Library, IBM (TSM) Tivoli Storage Manager
SAN Switches/Fabrics: Cisco MDS 9509, Brocade DCX, 48000
Protocols: TCP/IP, DHCP, DNS, HTTP, HTTPS, SSH, FTP, NFS, LDAP
Management Software (Tools): Centrify, ServiceNow, SolarWinds, Nagios, VMware VSphere/VCenter, CA Service Desk, CA eHealth, WSUS, Active directory, Cisco Anyconnect VPN, GlobalProtect VPN
IBM Storage: DSCLI, TPC, DS Storage Manager, Global Mirror/Flash Copy, HMC:
EMC Storage: SYMCLI, Navisphere/Unispehere Manager, Symmetrix SRDF/S, TimeFinder, OpenReplicator
PROFESSIONAL EXPERIENCE:
Confidential, PA
Hadoop Administrator
Responsibilities:
- Implementing, managing and supporting of Big Data Hadoop components/roles and CDH managed services.
- Optimizing performance in CDH, solved performance problems and apply best practices in the cluster configurations.
- Supporting Hadoop cluster implementation and ongoing Hadoop infrastructure administration using Cloudera Manager.
- Optimize and tune Yarn, Hive and Impala in the CDH clusters.
- Manage resource allocation and scheduling across the Hadoop cluster.
- Enable and deploying continues High Availability for HDFS and other CDH components.
- Manage Backup and disaster recovery of clusters using Cloudera Manager.
- Create HDFS and HBase Snapshots and replication using Cloudera Manager.
- Manage roles and services related to HDFS, Hive, Impala, Yarn, Sqoop, Zookeeper, Flume, Kafka etc.
- Commissioning and Decommissioning master and worker Nodes in Hadoop clusters.
- Monitor the health of Cloudera deployment, hosts, services, events, logs, and diagnose issue.
- Secure Hadoop clusters and CDH applications for user authentication and authorization using Kerberos deployment.
- Control and enforce role based authorization for Hive, HDFS and Impala using Sentry.
- Installing and upgrading Sentry services for Impala, Hive and HDFS.
- Managing and reviewing Hadoop log files.
- Monitoring and debugging Hadoop MapReduce jobs and applications running in Hadoop clusters.
- Install, configure and setup Hadoop CDH clusters for development, test and production environment.
- Importing and exporting data into HDFS, Hive and Hbase using Sqoop.
- Manage and configure Kafka with Flume for real-time event processing application.
- Perform administration of AWS EC2 Instances and S3 buckets on AWS cloud.
- Migration of large scale of LINUX Logical Volumes (LVM) and restore the data using AWS snapshots.
- Used Nagios to monitor applications, services, operating systems, network protocols and network infrastructure.
Hadoop Systems Administrator
Responsibilities:
- Managing, monitoring and installing Hadoop Clusters.
- Designing and implementing appropriate backup and recovery procedures for CDH clusters.
- Designing, storage management, performance tuning, monitoring and scaling of Hadoop clusters.
- Commission, decommission and balance CDH cluster nodes.
- Integrate and enable Kerberos for securing CDH cluster.
- Handling commission and de-commission nodes backup and restore.
- Install, configure and manage HBase clusters to store huge datasets & random data access.
- Performed cluster co-ordination through Zookeeper.
- Doing performance tuning of Hadoop clusters and maintain the cluster regulation.
- Perform RedHat Enterprise LINUX administration including installation, maintenance, patch upgrading and monitoring.
- Perform rolling upgrade on CDH cluster node in a production environment.
- Applying security using Kerberos linking with Active Directory and/or LDAP.
- Monitor Hadoop cluster connectivity and security.
- Managing and monitoring of HDFS and disk space on cluster.
- Worked on performance tuning and troubleshooting of MapReduce jobs by analyzing and reviewing Hadoop log files.
- Loaded data from Linux file system to HDFS also used Sqoop to efficiently transfer data from Oracle to HDFS.
- Worked with other module teams to install operating systems, Hadoop updates, patches, version upgrades as required.
- Manage large amount of users in all LINUX servers.
- Handing Linux operations and production support for RedHat Enterprise Linux servers hosted on physical HP ProLiant and IBM Servers.
Systems Administrator
Responsibilities:
- Have been responsible for administering large, multi-site UNIX/LINUX server environments and operating systems, software package installation, upgrades, system integrity, security, disaster recovery and performance.
- Managing and allocating storage from IBM storage DS8000 series storage, EMC Symmetrix VMAX 10k, VNX 5500 storage boxes storage systems using Navisphere/Unisphere Manager and SYMCLI.
- Worked on IBM HMC (Hardware Management Console) in DC and DR site for configuration, Monitoring and Maintenance of DS8300 storage racks.
- Responsible for day to day EMC SYMMETRIX VMAX Storage and TSM (Tivoli Storage Manager) / CommVault Backup administration, monitoring, escalation, reporting and management as required by the managed storage services contract.
- Configuring and managing Cisco and Brocade Fabric environment for soft/hard Zoning.
- Single-Sign-on implementation experience with Centrify.
- Cisco MDS 9509 Multilayer Director SAN switches/fabrics used for zoning configuration.
- Take care about Data Center (DC) by ordering and upgrading necessary hardware, supporting RAIDs, maintaining servers and installing new ones.
- Working on Windows and Linux Systems and EMC Storage administration, monitoring, escalation, reporting and management as required by the managed storage services contract.
- Day to day administration of Commvault Backup for enterprise Linux Servers.
- Daily CommVault backup scheduling, monitoring and restoring of all more than 1000 servers data.
- Analysis for failed or missed backups, restoration from the Backups.
- Created users, manage user permissions, maintain User & File System quota on Redhat Linux.
- Installation & Configuration of Logical Volume Manager - LVM and RAID.
- Day to day provisioning of storage including (Storage device/LUN/Volume selection & creation, Fabric Zoning, LUN Masking & Mapping).
- Administration of environment running VMware ESXi Hosts and Virtual Machines.
- Creating new devices, mapping and unmapping devices, managing Meta devices, SRDF/S, SRDF/Asynchronous, Recoverpoint, TIMEFINDER, BCV & management through the SYMCLI and unisphere.
- Worked successfully towards improving and maintaining the Backup Success rate to > 98%.
- Worked with server teams to insure the configuration and installation of the proper drivers, firmware, and multipath drivers to support the SAN environment.
- Provide performance tuning and regular maintenance in order to minimize downtime and maximize performance.
Systems Engineer
Responsibilities:
- Provisioning of IBM storage including (Storage device/LUN/Volume selection & creation, Fabric Zoning, LUN Masking & Mapping).
- Installed Windows 2003 Server for the organization with over 15000 clients.
- Performing Primary Health Checks on all the SAN devices in the environment.
- Involved in Cisco SAN switch configuration for zoning.
- Daily reporting the IBM storage replication links between remote sites.
- Provide 24/7 support for the company including high level executive and their administrative assistance.
- Provide Remote Support and administration on network for internal and external clients using tools like Microsoft Remote Desktop Connection (RDP).
- Provide technical support by responding to telephone support calls or ticketing inquiries.
- Monitoring on daily basis regarding hardware failures likes storage DDMs, HMC, cables connections.
- Troubleshooting the SAN connectivity on different Operating Systems Windows & Linux.
- Installing, configuring and managing Cisco switches, routers, servers, desktop computers and printers.