We provide IT Staff Augmentation Services!

Hadoop Administrator/devops Engineer Resume

Illinois, ChicagO

SUMMARY

  • Around 10 years of experience including 3 years of extensive exposure in Big Data Hadoop Administration with Devops and 7 years in Linux/Unix/Windows Administration Technologies.
  • Possesses strong abilities in Designing, Planning, Building, Configuring, Administering, Troubleshooting, Maintenance, Performance Monitoring and Fine - tuning of large scale Hadoop production Clusters using Apache, Cloudera and Hortonworks, on Physical as well as Cloud AWS servers.
  • Total 10 years of IT experience including 3 years of experience as Hadoop Administrator with Devops and 7 years as Linux/Unix/Windows Administrator.
  • Experience in Capacity Planning, installation, configuration and support of Cloudera CDH and Hortonworks Clusters
  • Experience on Installation and Configuration of Hadoop Ecosystems - HDFS, YARN, Map Reduce, Hbase, Storm,Kafka, Ranger, Spark, Pig, Hive, Sqoop, Flume, Tez, Zookeeper, Oozie
  • Experience in setting up Hadoop High Availability (Name Node, Resource Manager, Hive, Hbase and Disaster Recovery.
  • Strong experience with Hadoop Security and Governance using Ranger, Kerberos, Security Concepts-Best Practices, Falcon.
  • Good experience in BI. data warehousing, analytics, and Database
  • Good experience with data analytics tools such as Splunk, Cognos, Tableau
  • Strong experience with cluster security tools such as Kerberos, Ranger and Knox.
  • Good Knowledge on Apache Spark, Spark- Streaming, Apache Kafka
  • Experience on product upgrades,rollbacks, and updating patch fixes between different product versions.
  • Experience in Commissioning and Decommissioning of nodes within a cluster.
  • Experience in job automation using Oozie, cluster coordination through Zookeeper and MapReduce job scheduling using Capacity Schedulers.
  • Experience in tool Integration, automation, configuration management in GIT, SVN, Jira platforms.
  • Experience in Setup monitoring and alerts for the Hadoop cluster, creation of dashboards, alerts, and weekly status report for uptime, usage, issue, etc.
  • Experience on setting Sqoop to ingest RDBMS data into Hive and vice versa.
  • Experience in setting up Flume and Kafka to ingest log data and HDFS.
  • Expertize on Hadoop Cluster Performance Tuning and Troubleshooting.
  • Experience setting up Hadoop Clusters on EC2 instances for Product POCs.
  • Experience on Red Hat Enterprise Linux Administration and Devops Tools Puppet, chef, Jenkins.
  • Monitor the cluster - jobs, performance and fine-tune when necessary using tools Ambari, Autosys, AppDynamics.
  • Design, implement, test and document performance benchmarking strategy for platform as well for each use cases.
  • Experience with the Continuous Integration and Continuous Deployment pipeline ecosystem including tools such as Maven, Gradle,Jenkins and Puppet configuration tools.
  • Hands on experience in AWS provisioning and good knowledge of AWS services like EC2, S3, VPC, IAM, ELB
  • Maintain hardware-level stability and availability, including all break/fix issues, hardware replacement, hardware modifications, and hardware/server configurations
  • Participate in a 24x7 on-call support rotation

TECHNICAL SKILLS

Hadoop/Big Data: HDFS, Map Reduce, Hbase, Kafka, Storm, Spark, Ranger, Nifi,Pig, Hive, Sqoop, Flume, Hue, Oozie, Zookeeper

Operating System: RHEL/SUSE/Centos/Ubuntu/Solaris/Aix/Windows

Cloud & Virtualization Technology: EC2,S3,SQS,Lambda,Autoscaling,Adding EBS Volume, RDS, Redshift, ELB, VPC, Security Groups, IAM roles, Policies, EMR and DynamoDB

Hardware: HP ProLiant SL4540 Gen8,DL560,DL580,BL660c Del PowerEdge R720XD rack server,M620 blade server IBM System x3650 M4 BD,M620 blade server Oracle Big Data Appliacne X4-2,Sun Oracle X4-2L servers

Devops Tools: Jenkins, Gradle, Chef, Ansible, Git, GITHUB, SVN, Dockers, Maven, Agile, Scrum

Programming languages: Python, Linux shell scripts

Web Servers: Web Logic, Web Sphere, Apache Tomcat

Network Protocols: TCP/IP, HTTP, DNS, DHCP,NTP,SFTP,LDAP,SMTP,FTP,Kerberose

Database: Oracle/MySQL/HBASE/Cassandra

PROFESSIONAL EXPERIENCE

Hadoop Administrator/Devops Engineer

Confidential, Illinois, Chicago

Responsibilities:

  • Responsible for Managing large scale Hadoop cluster environment, handling all Hadoop environment builds, including design, capacity planning, cluster setup, performance tuning and ongoing monitoring.
  • Implemented Large scale Hadoop (Hortonworks HDP 2.4 Stack) enterprise Data lake and HDF Nifi cluster for SIT,DEV, UAT, CERT and PROD Environment.
  • Upgrade Hortonworks Ambari and HDP Stack from 2.3 to 2.4 Version in Dev, DR and Prod Environment.
  • Responsible for provisioning and managing Hadoop clusters on public cloud environment Amazon Web Services (AWS) -EC2 for Product POCs.
  • Diligently teaming with the infrastructure, network, database, application and Platform teams to guarantee high data quality and availability.
  • Configured the Chef and Ansible for Hadoop package deployments and other configuration push
  • Configure Continuous Delivery Build Pipeline with Gradle, Jenkins and Chef.
  • Setting up High-Availability for NameNode, ResourceManager, HBase, Hive and Storm .
  • Implemented Data Ingestion Cluster with Kafka, STORM and HBase.
  • Implemented Secured Hadoop clusters using Kerberos and Ranger integration with AD/LDAP at an Enterprise level.
  • Setup Monitoring and alerts for the Hadoop cluster, creation of dashboards, alerts, and weekly status report for uptime, usage, issue, etc.
  • Design, implement, test and document performance benchmarking strategy for platform as well for each use cases.
  • Monitor job performances, file system/disk-space management, cluster and database connectivity, log files, management of backup/security and troubleshooting various user issues.
  • Experience with performance tuning (JVM, JMX, connection pooling) using JConsole, AppDynamics or similar profiling tools.
  • Created shell and python scripts for data push from Informatica/Teradata to S3.
  • Collaborate with various cross functional teams: Infrastructure, network, database, systems and Application for various activities: deployment new hardware/software, environment, capacity uplift etc.
  • Hands-on experience in diagnosing, troubleshooting various networking, hardware & Linux server's services issues and performing preventive maintenance.

Hadoop/Linux Administrator

Confidential

Responsibilities:

  • Responsible for installation, configuration, supporting and managing Hadoop Clusters using Cloudera distributions.
  • Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage & review log files.
  • Collaborating with application teams to install operating system and Hadoop updates, patches, version upgrades.
  • Configuring Flume for efficiently collecting, aggregating and moving large amounts of log data from many different sources to HDFS.
  • Secured production environments by setting up Linux users, setting up Kerberos principals .
  • Installed Oozie workflow engine to schedule Hive and PIG scripts.
  • Hands on experience in Zookeeper and ZKFC in managing and configuring in NameNode failure scenarios.
  • Planning, installation, configuration, management and troubleshooting of Red Hat Enterprise Linux platform for test development and Production servers
  • Monitor Linux Server for CPU utilization, Memory Utilization and Disk Utilization for performance monitoring.
  • Installs, upgrades and manages packages via RPM and YUM package management Managed Disks and File systems using LVM on Linux. and monitoring
  • Hands-on experience in diagnosing, troubleshooting various networking, hardware & Linux server's services issues and performing preventive maintenance.
  • Participate in a 24x7 on-call support rotation and off-hours maintenance windows.

Linux Administrator

Confidential

Responsibilities:

  • Experience administering, installing, configuring and maintaining Linux.
  • Experience installing, upgrading and configuring RedHat Linux 5.x, 6.x using Kickstart Servers and Interactive Installation.
  • Responsible for creating and managing user accounts, security, rights, disk space and process monitoring in Solaris, CentOS and Redhat Linux.
  • Configures DNS, NFS, FTP, remote access, and security management. Creates Linux Virtual Machines using VMware Virtual Center.
  • Installs, upgrades and manages packages via RPM and YUM package management Managed Disks and File systems using LVM on Linux. and monitoring
  • Solve production problems when needed 24x7 Develop and document best practices
  • Planning, installation, configuration, management and troubleshooting of Red Hat Enterprise Linux platform for test development and Production servers
  • Monitor Linux Server for CPU utilization, Memory Utilization and Disk Utilization for performance monitoring.
  • Maintain hardware-level stability and availability, including all break/fix issues, hardware replacement, hardware modifications, and hardware/server configurations
  • Worked with Linux, Oracle Database, and Network teams to ensure the smooth relocation of the servers.
  • Perform physical hardware installation and configuration according to project requirements
  • Provision and manage Amazon Web Services resources for Production, QA, and Development
  • Responsible for 24x7 Global on call support for production Issues

Linux/Unix Administrator

Confidential

Responsibilities:

  • Planned, installed, and administration of Linux, Unix and Database.
  • Administered of storage and Backup issues.
  • Enterprise storage including FibreChannel, SAN, and VERITAS Storage Foundation (VxVM)
  • Task automation with scripting and enterprise tools
  • Established and tested a business continuity / Disaster recovery strategy for UNIX hosts
  • Capacity planning and solution design
  • Monitoring / Remote administration / Reporting and alerting
  • Wrote documentation, managed team collaboration site, and developed new processes

Linux/Unix Administrator

Confidential

Responsibilities:

  • Handling the on-call and resolving the critical tickets after business hours.
  • Preparing the SLA justification for missed SLA ( Sev 1&2 only)
  • Handling the restoration of the files from TSM and Networker Backup.
  • Attending Bridge call for sev1 & sev2 issues & working till issues get resolved.
  • Working on ticketing process based on ITIL (IT Infrastructure Library). Working on Automated and Manual Tickets
  • .Replacement of H/W (Motherboard, Memory, Media-Drives, NIC Cards, HBA) by coordinating with onsite team/vendor. installing the patches on all the servers for every quarter as per customer OLA
  • Performance tuning and monitoring using netstat, iosstat, vmstat and sar.
  • Supported and administered Veritas Volume Manager and Veritas Cluster products.

Linux/Windows Administrator

Confidential

Responsibilities:

  • Configuring and troubleshooting of linux and solarismachines.
  • Configuring and troubleshooting LVM (linux) and SVM (solaris)
  • User administration(Add, Delete, Modify) on Linux and Solaris
  • Configuring and troubleshooting NIS client (linux and solaris)
  • Configuring and troubleshooting NFS (linux and solaris).
  • Configuring and troubleshooting APACHE, SAMBA servers.
  • Taking the backup of production servers using Tivoli storage manager
  • Configuring and troubleshooting of RAID(0,1 and 5)

Hire Now