PROFESSIONAL SUMMARY:

• Around 8 years of professional experience including 3+ years of Hadoop Administration and 4+ years as Linux Admin.
• Responsible for day - to-day activities which includes HDFS support and maintenance, Cluster maintenance, creation/removal of nodes, Cluster Monitoring/ Troubleshooting, Manage and review Hadoop log files, Backup and restoring, capacity planning.
• Hands on experience in installing, configuring, and using Hadoop ecosystem components like HDFS, MapReduce, HBase, Zookeeper, Oozie, Hive, Sqoop, Pig, Impala and Flume.
• Worked with Hadoop developers and operating system admins in designing scalable supportable infrastructure for Hadoop.
• Well versed with installation, configuration, managing and supporting Hadoop cluster using various distributions like Apache Hadoop, Cloudera-CDH and Hortonworks HDP.
• Experience in performing minor and major Upgrades of Hadoop Cluster (Hortonworks Data Platform 1.7 to 2.1, CDH 5.5.5)
• Very strong experience working with Ansible playbooks and Ansible tower for automating the tasks to execute across the cluster.
• Experienced in installation, configuration, supporting and monitoring 100+ node Hadoop cluster using Cloudera manager and Hortonworks distributions.
• Experience in deploying and managing the multi-node development, testing and production Hadoop cluster with different Hadoop components using Cloudera Manager and Hortonworks.
• Experience in managing and reviewing Hadoop log files using Splunk and performed Hive tuning activities.
• Experience in providing security for Hadoop Cluster with Kerberos.
• Experience in Benchmarking, Backup and Disaster Recovery of Name node Metadata.
• Experience in working with Flume to load the log data from multiple sources directly into HDFS.
• Experience in administering the Linux systems to deploy Hadoop cluster and monitoring the cluster.
• Experience on Commissioning, Decommissioning, Balancing, and Managing Nodes and tuning server for optimal performance of the cluster.
• Disk space and physical memory management and automate manual tasks.
• Experience in HDFS data storage and support for running map-reduce jobs.
• Optimizing performance of Hbase/Hive/Pig jobs.
• Experience on Designing, Planning, Administration, Installation, Configuring, Troubleshooting, performance monitoring and Fine-tuning of Cassandra cluster.
• Experience in Ranger, Knox configuration to provide the security for Hadoop services (hive, base, HDFs etc.).

TECHNICAL SKILLS:

Big Data Technologies: HDFS, Hive, Map Reduce, Cassandra, Pig, Hcatalog, Phoenix, Falcon, Scoop, Flume, Zookeeper, Mahout, Oozie, Avro, HBase, MapReduce, HDFS, Storm, CDH 5.3, CDH 5.5

Monitoring Tools: Cloudera Manager, Ambari, Nagios, Ganglia

Scripting Languages: Shell Scripting, Puppet, Scripting, Python, Bash, CSH.

Programming Languages: C, Java, SQL, and PL/SQL.

Front End Technologies: HTML, XHTML, XML.

Application Servers: Apache Tomcat, WebLogic Server, Web sphere

Databases: Oracle 11g, MySQL, MS SQL Server, IBM DB2.

NoSQL Databases: HBase, Cassandra, MongoDB

Operating Systems: Linux, UNIX, MAC, Windows NT / 98 /2000/ XP / Vista, Windows 7, Windows 8.

Networks: HTTP, HTTPS, FTP, UDP, TCP/TP, SNMP, SMTP.

Security: Kerberos, Ranger.

PROFESSIONAL EXPERIENCE:

Hadoop Administrator

Confidential, Palo Alto, CA

Responsibilities:

• Worked as Hadoop Admin and responsible for taking care of everything related to the clusters total of 100 nodes ranges from POC (Proof-of-Concept) to PROD clusters.
• Worked as admin on Cloudera (CDH 5.5.2) distribution for clusters ranges from POC to PROD.
• Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage & review log files.
• Day to day responsibilities includes solving developer issues, deployments moving code from one environment to other environment, providing access to new users and providing instant solutions to reduce the impact and documenting the same and preventing future issues.
• Adding/installation of new components and removal of them through Cloudera Manager.
• Collaborating with application teams to install operating system and Hadoop updates, patches, version upgrades.
• Monitoring and controlling local file system disk space usage, log files, cleaning log files with automated scripts.
• Level 2, 3 SME for current Big Data Clusters at the Client Site and set up standard troubleshooting technique.
• Created and maintained user's accounts, profiles, security, rights disk space and process monitoring.
• Implemented and Configured High Availability Hadoop Cluster (Quorum Based) for HDFS, IMPALA and SOLR.
• Extensively worked on Impala to compare processing time of Impala with Apache Hive for batch applications to implement the former in project. Extensively Used Impala to read, write and query the Hadoop data in HDFS.
• Prepared adhoc phoenix queries on Hbase.
• Created secondary index tables using phoenix on HBase tables
• Monitored workload, job performance and capacity planning using Cloudera Manager.
• Involved in Analyzing system failures, identifying root causes, and recommended course of actions.
• Interacting with Cloudera support and log the issues in Cloudera portal and fixing them as per the recommendations.
• Performed Disk Space management to the users and groups in the cluster.
• Imported logs from web servers with Flume to ingest the data into HDFS.
• Using Flume and Spool directory loading the data from local system to HDFS.
• Experience in Chef, Puppet or related tools for configuration management.
• Retrieved data from HDFS into relational databases with Sqoop.
• Parsed cleansed and mined useful and meaningful data in HDFS using Map-Reduce for further analysis Fine tuning hive jobs for optimized performance.
• Implemented custom interceptors for flume to filter data and defined channel selectors to multiplex the data into different sinks.
• Partitioned and queried the data in Hive for further analysis by the BI team.
• Extending the functionality of Hive and Pig with custom UDF s and UDAF's.
• Involved in extracting the data from various sources into Hadoop HDFS for processing.
• Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hbase database and Sqoop.
• Regular disk management like adding/replacing hard drives on existing servers/workstations, partitioning according to requirements, creating new file systems or growing existing one over the hard drives and managing file systems.
• Creating collections and configurations, Register a Lily HBase Indexer configuration with the Lily HBase Indexer Service.
• Worked with Phoenix, a SQL layer on top of HBase to provide SQL interface on top of No-SQL database.
• Creating and truncating HBase tables in hue and taking backup of submitter ID(s).
• Configuring, Managing permissions for the users in hue.
• Troubleshooting, debugging & fixing Talend specific issues, while maintaining the health and performance of the ETL environment
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.

Environment: HDFS, Map Reduce, Hive 1.1.0, Hue 3.9.0, Pig, Flume, Oozie, Sqoop, CDH5, Apache Hadoop 2.6, Spark, SOLR, Storm, Knox, Cloudera Manager, Red Hat, MySQL and Oracle.

Hadoop Administrator

Confidential, SUNNYVALE, CA

Responsibilities:

• Capturing data from existing databases that provide SQL interfaces using Sqoop.
• Implemented Hadoop stack and different big data analytic tools, migration from different databases to Hadoop.
• Processed information from Hadoop HDFS. This information will comprise of various useful insights that can be used in the decision making process. All these insights will be presented to the users in the form of Charts.
• Working on different Big Data technologies, good knowledge of Hadoop, Map-Reduce, Hive.
• Developed various POCs over Hadoop, Big data.
• Worked on deployments and automation task.
• Monitored job performances, file system/disk-space management, cluster & database connectivity, log files, management of backup/security and troubleshooting various user issues.
• Importing Data using Apache Phoenix with SQLLINE.PY and PSQL.PY
• Installed and configured Hadoop cluster in pseudo and fully distributed mode environments.
• Involved in developing the data loading and extraction processes for big data analysis.
• Worked on professional services engagements to help customers design, build clusters, applications, troubleshoot network, disk and operating system related issues.
• Worked with Puppet for automated deployments.
• Installed and configured local Hadoop Cluster with 3 nodes and set up 4 nodes cluster on EC2 cloud.
• Written MapReduce code to process and parsing the data from various sources and storing parsed data into HBase and Hive using HBase-Hive Integration.
• Created Hive, Phoenix, HBase tables and HBase integrated Hive tables as per the design using ORC file format and Snappy compression
• Work with HBase and Hive scripts to extract, transform and load the data into HBase and Hive.
• Continuous monitoring and managing of the Hadoop cluster.
• Analyzed the data by performing Hive queries and running Pig scripts to know user behavior.
• Installed Oozie workflow engine to run multiple Hive and Pig jobs.
• Developing scripts and batch job to schedule a bundle (group of coordinators) which consists of various Hadoop programs using Oozie.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports.

Environment: Hadoop2.5.2, HDFS, Map Reduce, Hive, Flume, Sqoop, Cloudera CDH4, HBase, Oozie, Pig, AWS EC2 cloud.

Hadoop Administrator

Confidential, Atlanta, GA

Responsibilities:

• Worked in implementing Hadoop with the AWS EC2 system using a few instances in gathering and analyzing data log files. Developed Use cases and Technical prototyping for implementing PIG, HDP, HIVE and HBASE.
• Installed and configured Hortonworks Distribution Platform (HDP 2.3) on Amazon EC2 instances.
• Hands on experience in upgrading the cluster from HDP 2.0 to HDP 2.3
• Managing and reviewing Hadoop and HBase log files
• Proven results-oriented person with a focus on delivery
• Built and configured log data loading into HDFS using Flume.
• Provisioning, installing, configuring, monitoring, and maintaining HDFS, Yarn, HBase, Flume, Sqoop, Oozie, Pig, Hive, Ranger, Falcon, Smart sense, Storm, Kafka.
• Recovering from node failures and troubleshooting common Hadoop cluster issues.
• Scripting Hadoop package installation and configuration to support fully-automated deployments.
• Supporting Hadoop developers and assisting in optimization of map reduce jobs, Pig Latin scripts, Hive Scripts, and HBase ingest required.
• Implemented Kerberos for authenticating all the services in Hadoop Cluster.
• System/cluster configuration and health check-up.
• Continuous monitoring and managing the Hadoop cluster through Ambari.
• Hands on experience installing, configuring, administering, debugging and troubleshooting Apache and Datastax Cassandra clusters.

Environment: Hadoop, HDP 2.3, HDFS, Map Reduce, Impala, Sqoop, HBase, Hive, Flume, Oozie, Zoo keeper, solr, Performance tuning, cluster health, monitoring security, Shell Scripting, NoSQL/HBase/Cassandra, Cloudera Manager.

Hadoop Administrator

Confidential, Tampa, FL

Responsibilities:

• Worked in implementing Hadoop with the AWS EC2 system using a few instances in gathering and analyzing data log files. Developed Use cases and Technical prototyping for implementing PIG, HDP, HIVE and HBASE.
• Worked as a lead on Big Data Integration and Analytics based on Hadoop, SOLR and web methods technologies. Setting up and supporting Cassandra (1.2) /DataStax (3.2) for POC and prod environments using industry's best practices.
• Developed data pipeline using Flume, Sqoop, Pig and Java map reduce to ingest customer behavioral data and financial histories into HDFS for analysis.
• Tuned the Hadoop Clusters and Monitored for the memory management and for the Map Reduce jobs, to enable healthy operation of Map reduce jobs to push the data from SQL to NoSQL store.
• Hands on experience installing, configuring, administering, debugging and troubleshooting Apache and Datastax Cassandra clusters.
• Built, Stood up and delivered HADOOP cluster in Pseudo distributed Mode with NameNode, Secondary Name node, Job Tracker, and the Task tracker running successfully with Zookeeper installed, configured and Apache Accumulo ( NO SQL Google's Big table) is stood up in Single VM environment.
• Hands on experience installing, configuring, administering, debugging and troubleshooting Apache and Datastax Cassandra clusters.
• Used the Spark - Cassandra Connector to load data to and from Cassandra.
• Involved in scheduling Oozie workflow engine to run multiple Hive and pig jobs.

Environment: Hadoop, HDFS, Map Reduce, Impala, Splunk, Sqoop, HBase, Hive, Flume, Oozie, Zoo keeper, solr, Performance tuning, cluster health, monitoring security, Shell Scripting, NoSQL/HBase/Cassandra, Cloudera Manager.

Linux/Unix Administrator

Confidential

Responsibilities:

• Responsible for monitoring overall project and reporting status to stakeholders.
• Developed project user guide documents which help in knowledge transfer to new testers and solution repository document which gives quick resolution of any issues occurred in the past thereby reducing the number of invalid defects.
• Identify repeated issues in production by analyzing production tickets after each release and strengthen the system testing process to arrest those issues moving to production to enhance customer satisfaction
• Designed and coordinated creation of Manual Test cases according to requirement and executed them to verify the functionality of the application.
• Responsible for preventive maintenance of the servers on monthly basis. Configuration of the RAID for the servers. Resource management using the Disk quotas.
• Responsible for change management release scheduled by service providers.
• Generating the weekly and monthly reports for the tickets that worked on and sending report to the management.
• Established/implemented firewall rules, Validated rules with vulnerability scanning tools.
• Proactively detecting Computer Security violations, collecting evidence and presenting results to the management.
• Accomplished System/e-mail authentication using LDAP enterprise Database.
• Implemented a Database enabled Intranet web site using LINUX, Apache, MySQL Database backend.
• Installed Cent OS using Pre-Execution environment boot and Kick-start method on multiple servers. Monitoring System Metrics and logs for any problems.
• Running Cron-tab to back up Data. Applied Operating System updates, patches and configuration changes.

Environment: Windows 2008/2007 server, Unix Shell Scripting, SQL Manager Studio, Red Hat Linux, Microsoft SQL Server 2000/2005/2008, MS Access, NoSQL, Linux/Unix, Putty Connection Manager, Putty, SSH.