We provide IT Staff Augmentation Services!

Hadoop Admin Resume

3.00/5 (Submit Your Rating)

SUMMARY

  • 9 years of professional IT experiencewhich includes experience in Big Data ecosystem related technologies.
  • 5 years of exclusive experience in Hadoop Administration and its components like HDFS, Map Reduce, Apache Pig, Hive, Sqoop, Yarn, Oozie, Zookeeper and Flume.
  • Experience in installing Hadoop cluster using different distributions of Apache Hadoop, Cloudera and Hortonworks.
  • Proven expertise in Hadoop Projects Implementation and Configuring Systems.
  • Excellent experience in Hadoop architecture and various components such as Job Tracker, Task Tracker, NameNode, DataNode, MapReduce, YARN, Sqoop for data migration, Flume for data ingestion, Oozie for scheduling and Zookeeper for coordinating cluster resources
  • Wide experience in infrastructure layout, provisioning. Extensive experience in Amazon AWS cloud infrastructure with particular reference to provisioning, security, and management
  • Involved in Design and Development of technical specifications using Hadoop Echo System tools.
  • Administration, Testing, Hadoop administration activities such as installation and configuration and maintenance of clusters.
  • Good experience in developing MapReduce programs using Apache cloudera distribution, Centos & Windows.
  • Worthy experience in using Flume for gathering and transferring large amount of data from application server and also modeling variety of data.
  • Expertise in Commissioning, decommissioning, Balancing and Managing Nodes and tuning server for optimal performance of the cluster.
  • Involved inHadoopcluster environment administration that includes adding and removing cluster nodes, performance tuning, cluster Monitoring, Troubleshooting.
  • Strong understanding onHadooparchitecture and Map Reduce framework.
  • Experience in importing and exporting the data using Sqoop from HDFS to Relational Database systems/mainframe and vice - versa.
  • Good experience in data retrieving and processing using HIVE and PIG.
  • Hands on experience in analyzing Log files for Hadoop and eco system services and finding root cause.
  • Good working experience on Installing and maintaining the Linux servers.
  • Experience in analyzing data in HDFS through MapReduce, Hive, and Pig.
  • Supported Map Reduce Programs those are running on the cluster.
  • Configuration of Rack awareness for quick availability and processing of data.
  • Backup configuration and Recovery from a NameNode failure.
  • Good Experience in Planning, Installing and Configuring Hadoop Cluster in Apache Hadoop and Cloudera Distributions.
  • Hands on experience in Linux admin activities on RHEL & Cent OS.
  • Experience in deploying versions ofHadoop1.0 andHadoop2.0 (YARN).
  • Excellent command in creating Backups & Recovery and Disaster recovery procedures.
  • Excellent communication, interpersonal, analytical skills.
  • Ability to work independently as well as within a team environment.
  • Exceptional ability to learn new concepts. Hard working and enthusiastic.
  • Sound knowledge in SQL.
  • Familiar with UNIX commands.
  • Ability to rapidly grasp the new technologies and new concepts.

TECHNICAL SKILLS

BIG DATA TECHNOLOGIES: Hadoop, MapReduce, HDFS, Hive, Pig, Sqoop, HBase, Flume, Oozie, Zookeeper

HADOOP PLATFORMS: Cloudera, Hortonworks

PROGRAMMING LANGUAGES: C, Core Java, HTML, JavaScript.

JAVA/J2EE: Java 5/6, Eclipse, JDBC, Visual Studio.

DATABASE: MySQL, PostgreSql, Oracle, SQL Server, MS Access.

OPERATING SYSTEMS: Windows, Linux.

PROFESSIONAL EXPERIENCE

Hadoop Admin

Confidential

Responsibilities:

  • Working onHadoopHortonworks distribution which managed services viz. HDFSMapReduce2, Hive, Pig, Hbase, Sqoop, Flume, Ambari Metrics, YARN, Zookeeper, and Oozie etc. for 4 cluster ranges.
  • MonitorHadoopcluster connectivity and security on Ambari monitoring system.
  • Led the installation, configuration and deployment of product softwares on new edge nodes that connect and contactHadoopcluster for data acquisition.
  • Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodesTroubleshooting, Manage and review data backups, Manage & review log files.
  • Responsible for Installation and configuration of Hive, Pig, HBase and Sqoop on the Hadoop Cluster.
  • Day to day responsibilities includes solving developer issues, deployments moving code from one environment to other environment, providing access to new users and providing instant solutions to reduce the impact and documenting the same and preventing future issues.
  • Migrated data across clusters using DISTCP. Backing Up and Restoring HBase Data: Full shutdown backup using distcp, Using CopyTable to copy data from one table to another, exporting an HBase table to dump files on HDFS, Restoring HBase data by importing dump files from HDFS, Backing up NameNode metadata.
  • Monitored workload, job performance and capacity planning using Ambari. Collaborating with application teams to install operating system andHadoopupdates, patches, version upgrades.
  • Performed data analytics in Hive and then exported this metrics back to Oracle Database using Sqoop. Installed and configured Hive, Pig, Sqoop and Oozie on the HDP 2.2.0 cluster.
  • Involved in Analyzing system failures, identifying root causes, and recommended course of actions.
  • Assigning number of mappers and reducers to Map reduce cluster.
  • Using Flume, loading the data from local system to HDFS.
  • Retrieved data from HDFS into relational databases with Sqoop.
  • Setting up HBase cluster which includes master and region server configuration, High availability configuration, performance tuning and administration.
  • Fine tuning hive jobs for optimized performance.
  • Implemented custom interceptors for flume to filter data and defined channel selectors to multiplex the data into different sinks.
  • Experience designing data queries against data in the HDFS environment using tools such as Apache Hive. Involved in extracting the data from various sources intoHadoopHDFS for processing.

Environment: Hadoop HDFS, MapReduce, HIVE, PIG, FLUME, OOZIE, Sqoop, Eclipse, Hortonworks, Ambari, Redhat, Zookeeper, MYSQL

Hadoop Admin

Confidential

Responsibilities:

  • Administered Cluster maintenance, commissioning and decommissioning Data nodes, Cluster Monitoringtroubleshooting. Performed Adding/removing new nodes to an existingHadoopcluster.
  • Implemented Backup configurations and Recoveries from a NameNode failure. Monitored systems and services, architecture design and implementation ofHadoopdeploymentconfiguration management, backup, and disaster recovery systems and procedures.
  • Configured various property files like core-site.xml, hdfs-site.xml, mapred-site.xml based upon the job requirement.
  • Performed Importing and exporting data into HDFS using Sqoop. Installed variousHadoopEcosystems andHadoopDaemons. Implemented HDFS snapshot feature.
  • Installed and configured HDFS, Zookeeper, Map Reduce, Yarn, HBase, Hive, Scoop, Ansible and Oozie. Cluster coordination services through Zoo Keeper.
  • Involved in implementing High Availability and automatic failover infrastructure to overcome single point of failure for Namenode utilizing zookeeper services.

Environment: Hadoop, HDFS, Zookeeper, Map Reduce, YARN, HBase, Hive, Ansible, Sqoop, Oozie, Linux- Centos, Red Hat, Big Data Cloudera CDH, ApacheHadoop, SQL plus.

Hadoop Admin

Confidential

Responsibilities:

  • Install hadoop2/ Yarn, spark, scala IDE, JAVA JRE on three machines.
  • Configure these machines as a cluster, and set one Name node and two Data nodesand configuration of Hortonworks Data Platform HDP 2.2.0.
  • Worked on installing production cluster, commissioning & decommissioning of Data Nodes, Name Node recovery, capacity planning, and slots configuration.
  • Responsibilities include software upgrades, backup and recovery, cluster setup, monitoring and maintaining cluster performance.
  • Developed framework to import and export data from various sources like Teradata, Oracle, SQL server andFlat-files into HDFS.

Environment: Hortonworks HDP2.2x, Ambari, Oozie 4.2, Sqoop1.4.6, Mapreduce2, Sql Developer, Teradata, SSH, Eclipse, Jdk 1.7, CDH 3.x, 4.X, 5.X, Cloudera Manager 4&5,AWS,Pig, Hive, Flume, Kafka, Impala, CentOS

Hadoop admin

Confidential

Responsibilities:

  • Monitored workload, job performance and capacity planning using Cloudera Manager. Involved in extracting the data from various sources intoHadoopHDFS for processing.
  • Installed and configured Flume, Hive, Pig, Sqoop, HBase and Oozie on theHadoopcluster. Commissioned DataNodes when data grew and decommissioned when the hardware degraded.
  • Designed, configured and managed the backup and disaster recovery for HDFS data. Effectively used Sqoop to transfer data between databases and HDFS.
  • Handled importing data from various data sources, performed transformations using Hive, Map Reduce, and loaded data into HDFS.

Environment: JDK1.7, Redhat Linux, HDFS, Map-Reduce, Hive, Pig, Sqoop, Flume, Oozie, DB2, Hbase.

Java/ Hadoop admin

Confidential

Responsibilities:

  • Actively participated in requirements gathering, analysis, design, and testing phases. Designed use case diagrams, class diagrams, and sequence diagrams as a part of Design Phase and Object Model using UML.
  • User input validations done using JavaScript and developed use cases using UML. Used JavaScript for the client side validations and to provide the event driven programming with html files.
  • Coding using Java, Java Script and HTML. Developed SQL queries and stored procedures.
  • Developed use cases using UML according to business requirements.
  • Used JDBC to connect the J2EEserver with the relational database to provide database connectivity to database tables in Oracle.

We'd love your feedback!