We provide IT Staff Augmentation Services!

Hadoop Administrator Resume

PROFESSIONAL SUMMARY:

  • 8+ years of experience in IT specializing in Hadoop Administration, DevOps and Software testing.
  • Hands on experience in installation, configuration, management and support of full stack Hadoop Cluster both on premise and cloud using Hortonworks and Cloudera bundles.
  • Hands on Experience on Data Extraction, Transformation, Loading Data, Data Visualization using Hortonworks Platform HDFS, Hive, Sqoop, Hbase, Oozie, Beeline, Yarn, Spark, Scala
  • Good understanding on cluster capacity planning and configuring the cluster components based on requirements and job submission
  • Exposure to configuration of high availability cluster.
  • Knowledge of securing the Hadoop cluster using Kerberos and Sentry.
  • Experience in performance tuning of Hadoop cluster using various JVM metrics
  • Experience in writing Shell scripts using bash for process automation of databases, applications, backup and scheduling

TECHNICAL SKILLS:

Distribution Frameworks: Hadoop

Hadoop Distributions: Hortonworks, Cloudera

Hadoop Technologies: Hbase, Hive, Sqoop, Oozie, Spark, Yarn, Zookeeper

Data Analytics: Zeppelin, JupyterHub

Monitoring: Nagios, Grafana, OpenTSDB

Security: Kerberos, SSL Encryption

RDBMS: Oracle 9i, 10g, MySQL

Hadoop Framework: HADOOP SPRING

Programming Languages: JAVA & J2EE

JAVA Frameworks: MVC, Struts, Spring

Tools: Eclipse, Maven, Jenkins, ANT, Ansible

Web/Application servers: Apache Tomcat, WebLogic, JBoss

Operating Systems: RedHat Linux, Ubuntu Linux and Windows XP/Vista/7/8/10

Cloud Technologies: Microsoft Azure Insight

PROFESSIONAL EXPERIENCE:

Confidential

Hadoop Administrator

Platform: RHEL 6.X

Environment: Hadoop (1.X to 2.6), Java (jdk1.7,1.8)Duration :

Responsibilities:

  • Setup Nagios for Cluster Alerts
  • Forwarded below logs to Splunk Dashboard
  • Yarn Container’s
  • Hadoop Demon’s
  • Deployed JAVA App to forward the cluster metrics to OpenTSDB
  • Configured Grafana with OpenTSDB to visualize the data in Graphs
  • NNbench, MRbench, dfsio, DNbench, Hive TCP - DS, TCPH.
  • Confidential (Hbase) in Cloudera/Hortonworks clusters
  • Integrated Zeppelin with LDAP in CDH cluster
  • Configured Zeppelin Interpreters to process HDFS data.
  • JupyterHub Setup
  • Spark, Tez & MapReduce Engines for data processing
  • Developed MapReduce jobs in java for data preprocessing
  • OS Upgrade: Kernel Patch upgrade
  • Installing & configuring the HDP Hortonworks 2.x and Cloudera (CDH 5.5.1) Clusters in Dev and Production Environments

Confidential

Hadoop Administration

Platform: RHEL 6.X

Environment: Hadoop (1.X to 2.6), Java (jdk1.7,1.8)

Responsibilities:

  • Installed and configured Hadoop MapReduce, HDFS, Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
  • Installed multiple HDP 2.2 in 1500 Node plus cluster
  • Upgrading 1000 plus node clusters from 1.x Hadoop to 2.2
  • Worked on Capacity planning for the Production Cluster
  • Loading data from UNIX file system to HDFS.
  • Worked on taking Snapshot backups for HBase tables.
  • Good knowledge on fixing the cluster issues.
  • Involved in Cluster Monitoring backup, restore and troubleshooting activities.
  • Responsible for implementation and ongoing administration of Hadoop infrastructure
  • Managed and reviewed by sending Hadoop daemon logs to Splunk for deeper analysis
  • Worked on configuring queues in Oozie scheduler and observer oozie performance in grafana dashboards
  • Worked on Performance tuning in the Cluster level
  • Worked on On-boarding Users in the clusters
  • Production support which involves solving the user incidents varies from sev1 to sev5.
  • Provided security to the cluster with AMBARI ranger and created policies for each team to maintain security of project data between different teams in hadoop cluster.
  • Provided security to the cluster with AMBARI ranger and created policies for each team to maintain security of project data between different teams in hadoop cluster.
  • Worked on integration of tools like,Tableau with Hadoop .
  • Experienced in managing and reviewing Hadoop log files.
  • Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage & review log files
  • Changing the configurations based on the requirements of the users for the better performance of the jobs.
  • Investigate the root cause of Critical and P1/P2 tickets.
  • Experience in setting up Test, QA, and Prod environment. Written Pig Latin Scripts to analyze and process the data.
  • Involved in loading data from UNIX file system to HDFS. Created root cause analysis (RCA) efforts for the high severity incidents.
  • Implemented Cluster Security using Kerberos and HDFS ACLs.

Confidential

Hadoop Developer

Platform: RHEL 6.X

Environment: Apache Hadoop(2.0.0-mr1-cdh4.4.0 )

Responsibilities:

  • Worked as Hadoop Developer & responsible for writing MR jobs using Hadoop-Spring framework.
  • Developed Reducers & Mappers for Abinitio components functionalities
  • Implemented Distributed Cache for Application
  • Implemented Abinitio components in Apache Hadoop
  • Migrated a part of application to Apache Spark for performance analytics
  • Taking care of clusters total of 100 nodes ranges from POC to PROD clusters.
  • Set up Hortonworks cluster and installing all the ecosystem components through Ambari as well as manually from command line.
  • Cluster upgrade and patching without any data loss and with proper backup
  • Provided security to the cluster with AMBARI ranger and created policies for each team to maintain security of project data between different teams in hadoop cluster.
  • Worked on integration of tools like,Tableau with Hadoop .
  • Experienced in managing and reviewing Hadoop log files.
  • Mysql databases creation and setting up the users and maintain the backup of databases.
  • Data import/export from and to Hadoop from data oracle and other data sources.
  • Helping the users in production deployments throughout the process.
  • Managed and reviewed Hadoop Log files as a part of administration for troubleshooting purposes.
  • As a admin followed standard Back up policies to make sure the high availability of cluster.
  • Involved in Analyzing system failures, identifying root causes, and recommended course of actions. Documented the systems processes and procedures for future references.
  • Worked with systems engineering team to plan and deploy new Hadoop environments

Confidential

J2EE Developer

Platform: Windows 7

Environment: Struts(version: 1.1.2)

Responsibilities:

  • Helping the users in production deployments throughout the process.
  • Maintained Instructor-Led Training Module
  • Implement Learning Program Status for Application
  • Extensive experience in developing JAVA based applications using JAVA, J2EE, JDBC, EJB, Servlets, JSP, JSF, JMS and Webservices.
  • Designing and implementing the User Interface using JavaScript, HTML, Ext JS and AJAX.
  • Developed several REST web services which produces both XML and JSON to perform tasks, leveraged by both web and mobile applications
  • Created and configured new JIRA projects and worked with departments to maintain existing JIRA projects.
  • Designed and developed the application using Agile methodology and followed TDD, Scrum, pair programming
  • Designed dynamic and browser compatible pages using Jquery , JavaScript and Angular JS .

Confidential

J2EE Developer

Platform: Windows 7

Environment: Struts(version: 1.1.2)

Responsibilities:

  • Helping the users in production deployments throughout the process.
  • Maintained Instructor-Led Training Module
  • Implement Learning Program Status for Application
  • Java Full Stack Developer with experience in full software development life cycle including requirement analysis, design, coding, testing, debugging and maintenance.
  • Extensive experience in developing Web interfaces using HTML5, CSS3, Bootstrap, SASS, LESS, JavaScript, jQuery, Angular-JS and Backbone-JS.
  • Very good understanding of Object Oriented Design (OOD), Design Patterns, J2EE Design Patterns, MVC, Struts multi-tier architectures and distributed architectures.
  • Used spring dependency injection, annotations and Spring MVC components to implement business layer and navigation part of application
  • Developed Maven scripts and developed builds using Maven and continuous integration servers like Jenkins.
  • Responsible for configuring and deploying the builds on Tomcat Web Server.
  • Involved in the Performance Tuning of the Application on client side

Hire Now