SUMMARY:

• Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop HDFS, Yarn, MapReduce, HBase, Oozie, Hive, Sqoop, Pig, Flume, Storm, Kafka, Ranger, Falcon and Knox.
• Experience in deploying Hadoop cluster on Public and Private Cloud Environment like Cloudera, Hortonworks, and Amazon AWS.
• Setting up automated 24x7 monitoring and escalation infrastructure for Hadoop cluster using Nagios and Ganglia.
• Experience in managing and reviewing Hadoop log files.
• Experience in setting up the High - Availability Hadoop Clusters.
• Ability to prepare documents including Technical Design, Testing strategy, and supporting documents.

TECHNICAL SKILLS:

Programming Languages: Core Java, C++

Distribution Frameworks: Hadoop

Hadoop Distributions: Hortonworks, Cloudera

Hadoop Technologies: MapReduce, Hbase, Hive, Sqoop, Pig, Oozie

J2EE Components: Servlets, JSP.

Frame works: Hibernate.

Operating Systems: Windows, Linux and Unix

RDBMS: Oracle, MySQL

Web/Application Servers: Tomcat, Weblogic

PROFESSIONAL EXPERIENCE:

Confidential, Boca Raton, FL

Hadoop Administrator

Responsibilities:

• Designed/Installed/Configured/Maintained HDP2.4 cluster for application Development/Production.
• Designing a multi-node clusters for production environment based on the future data growth.
• Sizing of the cluster exercise performed along with stake holders to understand the data ingestion pattern and provided recommendations.
• Designing and implementing the non-production multi node environments.
• Upgrading Clusters from HDP2.1 to HDP2.3
• Responsible for Cluster maintenance, Adding and removing cluster nodes, Cluster Monitoring, troubleshooting, manage and review data backups, and manage and review Hadoop log files.
• Loading data from SAP DSO to Hadoop environment using Sqoop.
• Developed Hive queries and created necessary views to implement update process.
• Responsible for Hadoop Cluster monitoring using the tools like Nagios, Ganglia and Ambari.
• Worked with development teams to deploy Oozie workflow jobs to run multiple Hive and Pig jobs which run independently with time and data availability.
• Wrote the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
• Deployed a Kafka cluster with a separate zookeeper to enable processing of data using spark streaming in real-time and storing it in HBase.
• Implemented Capacity scheduler to securely share the available resources among multiple groups.
• Setting up of HDFS quota and resource quotas for different groups in a multi-tenant environment.
• Analyzed the data using Pig and written Pig scripts by grouping, joining and sorting the data
• Secured Hadoop Cluster by implementing Kerberos with Active Directory.
• Integrated Active Directory for authorization of users and groups of the system using Ranger and also implemented the perimeter security by using Apache Knox.
• Collected the logs data from web servers and integrated in to HDFS using Flume.

Environment: Hadoop, HBase, HDFS, Hive, Java, Pig, Zookeeper, Oozie, Flume.

Confidential, Fort Lauderdale, FL

Splunk Engineer

Responsibilities:

• Installed, Configured, Maintained, Tuned and Supported Splunk Enterprise Server 6.0 and Splunk Universal Forwarder 6.0.
• Administered a complex cluster based environment involving search heads in a cluster while the indexers are in standalone mode.
• Configured Splunk forwarder to send unnecessary log events to " Confidential " using props and transforms configurations.
• Created and configured management reports and dashboards in Splunk for application log monitoring.
• Active monitoring of Jobs through alert tools and responding with certain action to logs analyses the logs and escalate to high level teams on critical issues.
• Responsible for developing Splunk queries and dashboards targeted at understanding application performance and capacity analysis.
• Extensive experience on setting up the Splunk to monitor the customer volume and track the customer activity.
• Have involved as a Splunk Admin in capturing, analyzing and monitoring front end and middle ware applications.
• Created Splunk app for Enterprise Security to identify and address emerging security threats using continuous monitoring, alerting and analytics.
• Created and configured management reports and dashboards in Splunk for application log monitoring.
• Responsible for administering, maintaining, and configuring a 24 x 7 highly available, Splunk apps for production portal environment.
• Work closely with Application Teams to create new Splunk dashboards for Operation teams using advance XML and CSS.
• Created Shell Scripts to install Splunk Forwarders on all servers and configure with common configuration files such as Bootstrap scripts, Outputs.conf and Inputs.conf files.
• Extensively used Splunk Search Processing Language (SPL) queries, Reports, Alerts and Dashboards.
• Installation and implementation of the Splunk App for Enterprise Security and documented best practices for the installation and performed knowledge transfer on the process.
• Using DB connect for real-time data integration between SplunkEnterprise and databases.
• Analyzing in forwarder level to mask the customer sensitive data able to manage distributed search across set of indexers.
• Responsible to filter the unwanted data in heavy forwarder level thereby reducing the license cost.
• Worked with administrators to ensure Splunk is actively, accurately running, and monitoring on the current infrastructure implementation.
• Worked on properly creating/maintaining/updating necessary documentation for Splunk Apps, dashboards, upgrades and tracked issues.
• Provided On-call support for various production applications.
• Administered various shell and Python scripts for monitoring and automation.
• Extensive experience on setting up the Splunk to monitor the customer volume and track the customer activity.

Confidential, Miami, FL

Hadoop Administrator

Responsibilities:

• Installed and configured Hadoop on a cluster.
• Written multiple java based MapReduce jobs for data cleaning and preprocessing.
• Experienced in defining job flows using Oozie
• Experienced in managing and reviewing Hadoop log files
• Load and transform large sets of structured, semi structured and unstructured data
• Responsible to manage data coming from different sources and application
• Supported Map Reduce Programs those are running on the cluster
• Involved in loading data from UNIX file system to HDFS.
• Installed and configured Hive and also written Hive Confidential .
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
• Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage & review log files.
• Day to day responsibilities includes solving developer issues, deployments moving code from one environment to other environment, providing access to new users and providing instant solutions to reduce the impact and documenting the same and preventing future issues.
• Adding/installation of new components and removal of them through Ambari.
• Collaborating with application teams to install operating system and Hadoop updates, patches, version upgrades.
• Monitored workload, job performance and capacity planning
• Involved in Analyzing system failures, identifying root causes, and recommended course of actions.