SUMMARY

• Around 8+ years of professional experience includin 5+ years of Hadoop Administration and 3+ years as Linux Admin.
• Responsible for day - to-day activities which includes HDFSsupport and maintenance, Clustermaintenance, creation/removal of nodes, Cluster Monitoring/ Troubleshooting, Manage and review Hadoop log files, Backup and restoring, capacity planning.
• Hands on experience in installing, configuring, and using Hadoop ecosystem components like HDFS, MapReduce, HBase, Zookeeper, Oozie, Hive, Sqoop, Pig, Impala and Flume.
• Well versed with installation, configuration, managing and supporting Hadoop cluster using various distributions like Apache Hadoop, Cloudera-CDH and Hortonworks HDP.
• Experience in performing minor and major Upgrades of Hadoop Cluster (Hortonworks Data Platform 1.7 to 2.1, CDH 5.5.5)
• Worked on Containerization technology called Docker.
• Implemented AWS solutions using EC2, S3, RDS, EBS, Elastic Load Balancer, Auto scaling groups, AWS CLI.
• Experienced in installation, configuration, supporting and monitoring 100+ node Hadoop cluster using Cloudera manager and Hortonworks distributions.
• Worked in production phase for migrating the application from puppet to chef.
• Hands on experience in setting up the continuous integration tool Jenkins andBamboo.
• Experience in deploying and managing the multi-node development, testing and production Hadoop cluster with different Hadoop components using Cloudera Manager and Hortonworks.
• Experience in managing and reviewing Hadoop log files using Splunk and performed Hive tuning activities.
• Experience in Benchmarking, Backup and Disaster Recovery of Name node Metadata.
• Experience in working with Flume to load the log data from multiple sources directly into HDFS.
• Experience in administering the Linux systems to deploy Hadoop cluster and monitoring the cluster.
• Experience on Commissioning, Decommissioning, Balancing, and Managing Nodes and tuning server for optimal performance of the cluster.
• Experience in Chef, Puppet or related tools for configuration management.
• Experience in HDFS data storage and support for running map-reduce jobs.
• Optimizing performance of Hbase/Hive/Pig jobs.
• Experience on Designing, Planning, Administration, Installation, Configuring, Troubleshooting, performance monitoring and Fine-tuning of Cassandra cluster.
• Experience in Ranger, Knox configuration to provide the security for Hadoop services (hive, base, HDFs etc.).

TECHNICAL SKILLS

Big Data Technologies: HDFS, Hive, Map Reduce, Cassandra, Pig, Hcatalog, Phoenix, Falcon, Scoop, Flume, Zookeeper, Mahout, Oozie, Avro, HBase, MapReduce, HDFS, Storm, CDH 5.3, CDH 5.5

Monitoring Tools: Cloudera Manager, Ambari, Nagios, Ganglia

AWS Cloud Technologies: EC2, Elastic Beanstalk, SAAS, IAAS, Cloud Watch, Pivotal Cloud Foundry, S3, Route53, Redshift, DynamoDB, SQS, SES

Scripting Languages: Shell Scripting, Puppet, Scripting, Python, Bash, CSH.

Programming Languages: C, Java, SQL, and PL/SQL.

Front End Technologies: HTML, XHTML, XML.

Application Servers: Apache Tomcat, WebLogic Server, Web sphere

Databases: Oracle 11g, MySQL, MS SQL Server, IBM DB2.

NoSQL Databases: HBase, Cassandra, MongoDB

Operating Systems: Linux, UNIX, MAC, Windows NT / 98 /2000/ XP / Vista, Windows 7, Windows 8.

Networks: HTTP, HTTPS, FTP, UDP, TCP/TP, SNMP, SMTP.

Security: Kerberos, Ranger.

PROFESSIONAL EXPERIENCE

Sr. Hadoop Administrator

Confidential, New York, NY

Responsibilities:

• Worked as Hadoop Admin and responsible for taking care of everything related to the clusters total of 100 nodes ranges from POC (Proof-of-Concept) to PROD clusters.
• Worked as admin on Cloudera (CDH 5.5.2) distribution for clusters ranges from POC to PROD.
• Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage & review log files.
• Day to day responsibilities includes solving developer issues, deployments moving code from one environment to other environment, providing access to new users and providing instant solutions to reduce the impact and documenting the same and preventing future issues.
• Adding/installation of new components and removal of them through Cloudera Manager.
• Collaborating with application teams to install operating system and Hadoop updates, patches, version upgrades.
• Level 2, 3 SME for current Big Data Clusters Confidential the Client Site and set up standard troubleshooting technique.
• Monitored workload, job performance and capacity planning using Cloudera Manager.
• Involved in Analyzing system failures, identifying root causes, and recommended course of actions.
• Interacting with Cloudera support and log the issues in Cloudera portal and fixing them as per the recommendations.
• Imported logs from web servers with Flume to ingest the data into HDFS.
• Using Flume and Spool directory loading the data from local system to HDFS.
• Experience in Chef, Puppet or related tools for configuration management.
• Retrieved data from HDFS into relational databases with Sqoop.
• Parsed cleansed and mined useful and meaningful data in HDFS using Map-Reduce for further analysis Fine tuning hive jobs for optimized performance.
• Implemented custom interceptors for flume to filter data and defined channel selectors to multiplex the data into different sinks.
• Partitioned and queried the data in Hive for further analysis by the BI team.
• Extending the functionality of Hive and Pig with custom UDF s and UDAF's.
• Involved in extracting the data from various sources into Hadoop HDFS for processing.
• Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hbase database and Sqoop. .
• Creating collections and configurations, Register a Lily HBase Indexer configuration with the Lily HBase Indexer Service.
• Creating and truncating HBase tables in hue and taking backup of submitter ID(s).
• Configuring, Managing permissions for the users in hue.
• Responsible for building scalable distributed data solutions using Hadoop.
• Installed Oozie workflow engine to run multiple Hive and pig jobs.
• Troubleshooting, debugging & fixing Talend specific issues, while maintaining the health and performance of the ETL environment.
• Create, modify and execute DDL and ETL scripts for De-normalized tables to load data into Hive and AWS Redshift tables.
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.

Environment: HDFS, Docker, Puppet, Map Reduce, Hive 1.1.0, Hue 3.9.0, Pig, Flume, Oozie, Sqoop, CDH5, Apache Hadoop 2.6, Spark, AWS, SOLR, Storm, Knox, Cloudera Manager, Red Hat, MySQL and Oracle.

Hadoop Administrator

Confidential, SUNNYVALE, CA

Responsibilities:

• Capturing data from existing databases that provide SQL interfaces using Sqoop.
• Implemented Hadoop stack and different big data analytic tools, migration from different databases to Hadoop.
• Processed information from Hadoop HDFS. This information will comprise of various useful insights that can be used in the decision making process. All these insights will be presented to the users in the form of Charts.
• Working on different Big Data technologies, good knowledge of Hadoop, Map-Reduce, Hive.
• Developed various POCs over Hadoop, Big data.
• Worked on deployments and automation task.
• Migrated Cassandra, Hadoop cluster on AWS and defined different read/write strategies for geographies.
• Installed and configured Hadoop cluster in pseudo and fully distributed mode environments.
• Involved in developing the data loading and extraction processes for big data analysis.
• Worked on professional services engagements to help customers design, build clusters, applications, troubleshoot network, disk and operating system related issues.
• Worked with Puppet for automated deployments.
• Installed and configured local Hadoop Cluster with 3 nodes and set up 4 nodes cluster on EC2 cloud.
• Written MapReduce code to process and parsing the data from various sources and storing parsed data into HBase and Hive using HBase-Hive Integration.
• Work with HBase and Hive scripts to extract, transform and load the data into HBase and Hive.
• Continuous monitoring and managing of the Hadoop cluster.
• Analyzed the data by performing Hive queries and running Pig scripts to know user behavior.
• Installed Oozie workflow engine to run multiple Hive and Pig jobs.
• Developing scripts and batch job to schedule a bundle (group of coordinators) which consists of various Hadoop programs using Oozie.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports.

Environment: Hadoop2.5.2, HDFS, Map Reduce, AWS, Hive,Docker, Puppet, Flume, Sqoop, Cloudera CDH4, HBase, Oozie, Pig, AWS EC2 cloud.

Hadoop Administrator

Confidential, Atlanta, GA

Responsibilities:

• Client wanted to migrate from In-Premise cluster to the Amazon Web services Cloud (AWS), hence Built a Production and QA Cluster with the latest distribution of Hortonworks - HDP stack managed by Ambari on AWS Cloud.
• Worked in implementing Hadoop with the AWS EC2 system using a few instances in gathering and analyzing data log files. Developed Use cases and Technical prototyping for implementing PIG, HDP, HIVE and HBASE.
• Installed and configured Hortonworks Distribution Platform (HDP 2.3) on Amazon EC2 instances.
• Hands on experience in upgrading the cluster from HDP 2.0 to HDP 2.3
• Managing and reviewing Hadoop and HBase log files
• Proven results-oriented person with a focus on delivery
• Built and configured log data loading into HDFS using Flume.
• Provisioning, installing, configuring, monitoring, and maintaining HDFS, Yarn, HBase, Flume, Sqoop, Oozie, Pig, Hive, Ranger, Falcon, Smart sense, Storm, Kafka.
• Recovering from node failures and troubleshooting common Hadoop cluster issues.
• Scripting Hadoop package installation and configuration to support fully-automated deployments.
• Supporting Hadoop developers and assisting in optimization of map reduce jobs, Pig Latin scripts, Hive Scripts, and HBase ingest required.
• Implemented Kerberos for authenticating all the services in Hadoop Cluster.
• System/cluster configuration and health check-up.
• Continuous monitoring and managing the Hadoop cluster through Ambari.
• Hands on experience installing, configuring, administering, debugging and troubleshooting Apache and Datastax Cassandra clusters.

Environment: Hadoop, HDP 2.3, HDFS, Map Reduce, Impala, Sqoop, HBase, Hive, Flume, Oozie, Zoo keeper, solr, Performance tuning, AWS, cluster health, monitoring security, Shell Scripting, NoSQL/HBase/Cassandra, Cloudera Manager.

Hadoop Administrator

Confidential, Tampa, FL

Responsibilities:

• Worked in implementing Hadoop with the AWS EC2 system using a few instances in gathering and analyzing data log files. Developed Use cases and Technical prototyping for implementing PIG, HDP, HIVE and HBASE.
• Worked as a lead on Big Data Integration and Analytics based on Hadoop, SOLR and web methods technologies. Setting up and supporting Cassandra (1.2) /DataStax (3.2) for POC and prod environments using industry's best practices.
• Developed data pipeline using Flume, Sqoop, Pig and Java map reduce to ingest customer behavioral data and financial histories into HDFS for analysis.
• Tuned the Hadoop Clusters and Monitored for the memory management and for the Map Reduce jobs, to enable healthy operation of Map reduce jobs to push the data from SQL to NoSQL store.
• Hands on experience installing, configuring, administering, debugging and troubleshooting Apache and Datastax Cassandra clusters.
• Built, Stood up and delivered HADOOP cluster in Pseudo distributed Mode with NameNode, Secondary Name node, Job Tracker, and the Task tracker running successfully with Zookeeper installed, configured and Apache Accumulo( NO SQL Google's Big table) is stood up in Single VM environment.
• Hands on experience installing, configuring, administering, debugging and troubleshooting Apache and Datastax Cassandra clusters.
• Used the Spark - Cassandra Connector to load data to and from Cassandra.
• Involved in scheduling Oozie workflow engine to run multiple Hive and pig jobs.

Environment: Hadoop, HDFS, Map Reduce, Impala, Splunk, Sqoop, HBase, Hive, Flume, Oozie, Zoo keeper, solr, Performance tuning, cluster health, monitoring security, Shell Scripting, NoSQL/HBase/Cassandra, Cloudera Manager.

Linux/Unix Administrator

Confidential

Responsibilities:

• Responsible for monitoring overall project and reporting status to stakeholders.
• Developed project user guide documents which help in knowledge transfer to new testers and solution repository document which gives quick resolution of any issues occurred in the past thereby reducing the number of invalid defects.
• Identify repeated issues in production by analyzing production tickets after each release and strengthen the system testing process to arrest those issues moving to production to enhance customer satisfaction
• Designed and coordinated creation of Manual Test cases according to requirement and executed them to verify the functionality of the application.
• Responsible for preventive maintenance of the servers on monthly basis. Configuration of the RAID for the servers. Resource management using the Disk quotas.
• Responsible for change management release scheduled by service providers.
• Generating the weekly and monthly reports for the tickets that worked on and sending report to the management.
• Established/implemented firewall rules, Validated rules with vulnerability scanning tools.
• Proactively detecting Computer Security violations, collecting evidence and presenting results to the management.
• Accomplished System/e-mail authentication using LDAP enterprise Database.
• Implemented a Database enabled Intranet web site using LINUX, Apache, MySQL Database backend.
• Installed Cent OS using Pre-Execution environment boot and Kick-start method on multiple servers. Monitoring System Metrics and logs for any problems.
• Running Cron-tab to back up Data. Applied Operating System updates, patches and configuration changes.

Environment: Windows 2008/2007 server, Unix Shell Scripting, SQL Manager Studio, Red Hat Linux, Microsoft SQL Server 2000/2005/2008, MS Access, NoSQL, Linux/Unix, Putty Connection Manager, Putty, SSH.