SUMMARY:

• Around 5 years of experience in Hadoop Administration &Big Data Technologies
• Total 6+ years of experience in IT Industry.
• Experience with complete Software Design Lifecycle including design, development, testing and implementation of moderate to advanced complex systems.
• Hands on experience in installation, configuration, supporting and managing Hadoop Clusters using Apache, Cloudera (CDH4, CDH5), Hortonworks (HDP2.3, HDP2.4) Yarn distributions.
• Hands on experience on configuring a Hadoop cluster in a professional environment and on Amazon Web Services (AWS) using an EC2 instance.
• Hands on experience on high availability of Cluster.
• Experience of installation, performance tuning and monitoring of spark applications
• Experience of Security Setup in Hadoop cluster using Kerberos
• Configured Kafka and Storm for real time data ingestion.
• Hadoop Cluster capacity planning, performance tuning, cluster Monitoring, Troubleshooting.
• Design Big Data solutions for traditional enterprise businesses.
• Used Network Monitoring Daemons like Ganglia and Service monitoring tools like Nagios.
• Adding/removing new nodes to an existing hadoop cluster.
• Backup configuration and Recovery from a NameNode failure.
• Decommissioning and commissioning the Node on running hadoop cluster.
• Installation of various Hadoop Ecosystems and Hadoop Daemons.
• Installation and configuration of Sqoop and Flume.
• Involved in bench marking Hadoop/HBase cluster file systems various batch jobs and workloads
• Experience monitoring and troubleshooting issues with Linux memory, CPU, OS, storage and network
• Good experience on Design, configure and manage the backup and disaster recovery for Hadoop data.
• Hands on experience in analyzing Log files for Hadoop and eco system services and finding root cause.
• Experience on Commissioning, Decommissioning, Balancing, and Managing Nodes and tuning server for optimal performance of the cluster.
• As an admin involved in Cluster maintenance, trouble shooting, Monitoring and followed proper backup& Recovery strategies.
• Experience in HDFS data storage and support for running map - reduce jobs.
• Installing and configuring hadoop eco system like sqoop, pig, hive.
• Knowledge on Hbase and zookeeper.
• Experience in importing and exporting the data using Sqoop from HDFS to Relational Database systems/mainframe and vice-versa.
• Good knowledge of Java.
• Optimizing performance of Hbase/Hive/Pig jobs.
• Scheduling all hadoop/hive/sqoop/Hbase jobs using Oozie.

PROFESSIONAL EXPERIENCE:

Confidential, NC

Hadoop Admin

Responsibilities:

• Involved in start to end process of hadoop cluster setup over Cloudera manager where in installation, configuration and monitoring the Hadoop Cluster.
• Responsible for Cluster maintenance, commissioning and decommissioning Data nodes, Cluster Monitoring, Troubleshooting, Manage and review data backups, Manage & review Hadoop log files.
• Monitoring systems and services, architecture design and implementation of hadoop deployment, configuration management, backup, and disaster recovery systems and procedures.
• Configured various property files like core-site.xml, hdfs-site.xml, mapred-site.xml based upon the job requirement
• Involved in implementing High Availability and automatic failover infrastructure to overcome single point of failure for Namenode utilizing zookeeper services.
• Hands on experience in provisioning and managing multi-node Hadoop Clusters on public cloud environment Amazon Web Services (AWS) - EC2 and on private cloud infrastructure.
• Involved in cluster capacity planning, Hardware planning, Installation, Performance tuning of the Hadoop cluster.
• Importing and exporting data into HDFS using Sqoop.
• Experienced in define being job flows with Oozie.
• Loading log data directly into HDFS using Flume.
• Experienced in managing and reviewing Hadoop log files.
• Installation of various Hadoop Ecosystems and Hadoop Daemons.
• Installation and configuration of Sqoop Flume &Hbase
• Managed and reviewed Hadoop Log files as a part of administration for troubleshooting purposes. Communicate and escalate issues appropriately.
• Worked on Hadoop cluster and data querying tools Hive to store and retrieve data.
• Exported analyzed data to HDFS using Sqoop for generating reports.
• Worked on Oozie workflow engine to run multiple Map Reduce jobs.
• As an admin followed standard Back up policies to make sure the high availability of cluster.
• Involved in Analyzing system failures, identifying root causes, and recommended course of actions. Documented the systems processes and procedures for future references.
• Worked with systems engineering team to plan and deploy new hadoop environments and expand existing hadoop clusters.
• Monitored multiple hadoop clusters environments using Ganglia and Nagios. Monitored workload, job performance and capacity planning using Cloudera Manager.
• Installed and configured Flume, Hive, Pig, Sqoop and Oozie on the hadoop cluster.
• Involved in Installing and configuring Kerberos for the authentication of users and hadoop daemons.

Environment: Hadoop, HDFS, Hive, Sqoop, Flume, Zookeeper and HBase, Oracle 9i/10g/11g RAC with Solaris/redhat, Exadata Machines X2/X3, Big Data Cloudera CDH Apache Hadoop, Toad, SQL plus, Oracle Enterprise Manager (OEM), RMAN, Shell Scripting, Golden Gate, Redhat/SuseLinix, EM Cloud Control

Confidential, NC

Hadoop Admin

Responsibilities:

• Installed and configured Cloudera CDH 5.7.1 with Hadoop Eco-Systems like Hive, Oozie, Hue, Spark, kafka, Hbase, Yarn
• Configured AD, Centerify and integrated with Kerberos.
• Installed and configured Kafka Cluster
• Installed MySQL and MySQL Master - Slave setup.
• Installed Oozie workflow engine to run multiple Hive and Pig jobs.
• Commissioning Data Nodes when data grew and De-commissioning of data nodes from cluster in hardware degraded.
• Setting up and managing HA Name Node to avoid single point of failures in large clusters.
• Worked with different applications teams to integrate with Hadoop.
• Worked with data delivery teams to setup new Hadoop users, Linux users, setting up Kerberos principles and testing HDFS, Hive.
• Involved in cluster capacity planning, Hardware planning, Installation, Performance tuning of the Hadoop cluster.
• Hands on experience in provisioning and managing multi-node Hadoop Clusters on public cloud environment Amazon Web Services (AWS) - EC2 and on private cloud infrastructure.
• Responsible for loading the customer's data and event logs from Oracle database, Teradata into HDFS using Sqoop
• Involved in initiating and successfully completing Proof of Concept on SQOOP for Pre-Processing, Increased Reliability and Ease of Scalability over traditional Oracle database.
• End-to-end performance tuning of Hadoop clusters and Hadoop MapReduce routines against very large data sets.
• Involved in loading data from LINUX file system to HDFS.
• Importing and exporting data into HDFS and Hive using Sqoop and Flume.
• Reviewed the HDFS usage and system design for future scalability and fault-tolerance.
• Exported analyzed data to HDFS using Sqoop for generating reports.
• Cluster co-ordination services through Zookeeper.
• Analyze the log files and process through Flume
• Integrated Attunity and Cassandra with CDH Cluster.
• Worked closely with Data Center team and Linux team in configuring VM and linux boxes.
• Involved in and finalizing Cloudera SOW and MSA with Cloudera

Environment: RHEL 6.7, CentOS 7.2, Shell Scripting, Java (JDK 1.7), Map Reduce, Oracle, SQL server, Attunity, Cloudera CDH 5.7.x, Hive, Zookeeper and Cassandra.