Hadoop Consultant Resume

SUMMARY

AWS certified Solutions Architect - Associate and AWS certified Developer - Associate.
Over all 6+ years of working experience, including with 5+ years of experience as a Hadoop Administration and along with around 1+ Year of experience in Linux admin related roles.
Strong working experience with Big D Confidential and Hadoop Ecosystems including HDFS, PIG, HIVE, HBase, Yarn, Sqoop, Flume, Oozie, Hue, Map Reduce and Spark.
Deep cover in minor and major upgrades of Hadoop and Hadoop eco system.

TECHNICAL SKILLS

Big D Confidential Ecosystems: Hadoop, Map Reduce, HDFS, HBase, Zookeeper, Hive, Pig, Sqoop, Kafka, Cassandra, Oozie, Flume, Chukwa, Pentaho Kettle and Talend.

Cloud: AWS (EC2, S3, ELB, EBS, VPC, Auto Scaling), Azure

Programming Languages: Java, C/C++, eVB, Assembly Language (8085/8086)

Scripting Languages: JSP & Servlets, PHP, JavaScript, XML, HTML, Python and Bash

D Confidential bases: NoSQL, Oracle

UNIX Tools: Apache, Yum, RPM

Tools: Eclipse, JDeveloper, JProbe, CVS, Ant, MS Visual Studio

Platforms: Windows (2000/XP), Linux, Solaris, AIX, HPUX

Application Servers: Apache Tomcat 5.x 6.0, Jboss 4.0

Testing Tools: Net Beans, Eclipse, WSAD, RAD

Methodologies: Agile, UML, Design Patterns

PROFESSIONAL EXPERIENCE:

Hadoop Consultant

Confidential

Responsibilities:

Worked as Hadoop Admin and responsible for taking care of everything related to the clusters total of 100 nodes ranges from POC (Proof-of-Concept) to PROD clusters.
Installation, Configuration, up gradation and administration of Windows, Sun Solaris, RedHat Linux and Solaris.
Hands on experience in installation, configuration, management and support of full stack Hadoop Cluster both on premise and cloud using Horton works and Cloudera bundles.
Worked as admin on Cloudera (CDH 5.5.2) distribution for clusters ranges from POC to PROD.
Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning D Confidential nodes, Troubleshooting, Manage and review d Confidential backups, Manage & review log files.
Set up Hortonworks Infrastructure from configuring clusters to Node security using Kerberos.
Worked extensively on AWS Components such as Airflow, Elastic Map Reduce (EMR), Athena, and Snowflake.
Created and maintained various Shell and Python scripts for automating various processes.
Involved in developing custom scripts using Shell (bash, ksh) to automate jobs.
Installing MySQLDB in Linux and Customize the MySQL DB parameters.
Installed Kafka cluster with separate nodes for brokers.
Installing and configuring Kafka and monitoring the cluster using Nagios and Ganglia.
Responsible for installation, configuration and management of Linux servers and POC Clusters in the VMware environment.
Configuring Apache and supporting them on Linux production servers.
Involved in file movements between HDFS and AWS S3 and extensively worked with S3 bucket in AWS.
Experience in setting up Kafka cluster for publishing topics and familiar with lambda architecture.
Adding/installation of new components and removal of them through Cloudera Manager.
Collaborating with application teams to install operating system and Hadoop updates, patches, version upgrades.
Level 2, 3 SME for current Big D Confidential Clusters at the Client Site and set up standard troubleshooting technique.
Managed servers on the Amazon Web Services (AWS) platform instances using Puppet, Chef Configuration management.
Loaded log d Confidential into HDFS using Flume, Kafka and performing ETL integrations.
Experience with designing and building solutions for d Confidential ingestion both real time & batch using Sqoop/PIG/Impala/Kafka.
Involved in Analyzing system failures, identifying root causes, and recommended course of actions.
Interacting with Cloudera support and log the issues in Cloudera portal and fixing them as per the recommendations.
Worked extensively with importing metad Confidential into Hive using Python and migrated existing tables and applications to work on AWS cloud (S3).
Imported logs from web servers with Flume to ingest the d Confidential into HDFS.
Implemented Kerberos for authenticating all the services in Hadoop Cluster.
Parsed cleansed and mined useful and meaningful d Confidential in HDFS using Map-Reduce for further analysis Fine tuning hive jobs for optimized performance.
Worked on analyzing Hadoop cluster and different big d Confidential analytic tools including Pig, Hbase d Confidential base and Sqoop.
Installed Oozie workflow engine to run multiple Hive and pig jobs.
Troubleshooting, debugging & fixing Talend specific issues, while maintaining the health and performance of the ETL environment.
Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.

Environment: HDFS, Docker, Puppet, Map Reduce, Hive 1.1.0, Hue 3.9.0, Pig, Flume, Oozie, Sqoop, CDH5, Apache Hadoop 2.6, Spark, AWS, SOLR, Storm, Knox, Cloudera Manager, Red Hat, MySQL and Oracle.

We provide IT Staff Augmentation Services!

We'd love your feedback!

Resume Categories

Client Services

Job Seekers

Visa Sponsorship