SUMMARY

• Professional with 7+ years of IT experience in industries including Banking, Healthcare and Education.
• Cloudera Certified Hadoop administrator with 5+ years of experience in activities such as installation and configuration of clusters using Apache Hadoop, Cloudera (CDH), Hortonworks (HDP), AWS Elastic Map Reduce (EMR).
• Worked in complete Software Development Life Cycle (analysis, design, development, testing, implementation and support) using Agile Methodologies.
• Deployed Hadoop cluster on on-premises Data Centers and Private Cloud Environments.
• Hands on experience in installation, configuration, supporting and managing Hadoop Clusters using Apache, Cloudera (CDH3, CDH4), Yarn distributions (CDH 5.X).
• Hands on experience in installing, configuring, and using Hadoop ecosystem components likeHDFS, Yarn, MapReduce, Spark, HBase, Oozie, Hive, Sqoop, Flume, Storm, Kafka and Sentry.
• Installed and configured setting up of automated 24x7 monitoring and escalation infrastructure for Hadoop clusters using Nagios and Ganglia.
• Experienced on Commissioning, Decommissioning, Balancing, and Managing Nodes and tuning server for optimal performance of the cluster.
• Experienced in understanding the security requirements for Hadoop and integrating with Kerberos authentication infrastructure- KDC server setup, creating realm /domain.
• Experienced in creating service principles, user principles and establishing cross-realm trust.
• Strong knowledge on enforcing RBACto hive data and metadata on a Hadoop cluster using Sentry.
• Experienced in setting up the High-Availability Hadoop Clusters and BDR clusters.
• Hadoop Cluster capacity planning, performance tuning, cluster Monitoring, Troubleshooting.
• Good experience on configuring and managing the backup and disaster recovery for Hadoop data.
• Hands on experience in managing, reviewing and analyzing Log files for Hadoop and eco system services and finding root cause by performing root cause analysis.
• Involved in Analyzing system failures, identifying root causes, and recommended course of actions. Documented the systems processes and procedures for future references.
• Experienced in HDFS data storage and support for running map-reduce jobs and spark jobs.
• Experienced in importing and exporting the data using Sqoop from HDFS to RDBMS and Scheduling all Hadoop/Hive/Sqoop/HBase jobs using Oozie.
• Experienced on Rack awareness configuration for quick availability and processing of data.
• Involved in balancing the loads on server,tuning of server for optimal performance of the cluster.
• Handsome experience in Linux admin activitiessuch as user management, scheduling cron jobs, setting up the Linux environments, Password less SSH, creating file systems, disabling firewalls, Swappiness, Selinux and installing Java.
• Experienced in configuration management tools like puppet, Ansible and chef.
• Good understanding in Deployment of Hadoop Clusters Using Automated Puppet scripts
• Experienced in hardware recommendations, performance tuning and benchmarking
• Experienced in IP Management (IP Addressing, Sub-netting, Ethernet Bonding, Static IP)
• Flexible with Unix/Linux and Windows Environments working with OS like Centos 5/6, Ubuntu 10/11 and RedHat 6/7.
• Experienced in Linux Storage Management. Configuring RAID Levels, Logical Volumes.

TECHNICAL SKILLS

Scripting Language: Shell, bash

Hadoop Ecosystem: Hadoop 2.7.x, Spark 2.1.0, MapReduce, Hive 2.1.1, Sqoop 1.99.7, Flume 1.7.0, Kafka 2.1.0, Oozie 3.1.3, Yarn 0.21.3, Pig 0.14, Zookeeper 3.4.6

Database: MySQL 5.x, Oracle 11g, HBase 1.3.0, Cassandra 3.10, PL/SQL 11g, MS SQL Server.

Hadoop Distributions : Cloudera, Hortonworks, AWS EMR, Apache Hadoop

IDE Application: Eclipse 4.6, Net beans

Collaboration: Git 2.12.0, Scala Test 3.0.1

Operating Systems: Windows10, Mac OS, Ubuntu, Centos, Red hat

Data Analysis & Viz: Tableau

Cloud Environment: AWS EC2, S3, IAM, VPC, ROUTE53, Cloud watch, Cloudera Formation

Web Services: RESTful, SOAP

PROFESSIONAL EXPERIENCE

Confidential, San Ramon, CA

Hadoop Admin

Responsibilities:

• Provided infrastructure support for multiple clusters like Production(Prod), Pre-Production(Pre-prod), Quality (QA) and Disaster Recovery(DR)
• Installed and configured Hadoop cluster across various environments through Cloudera Manager
• Installed and configured MYSQL and Enabled High Availability.
• Installed and configured Sentry server to enable schema level Security.
• Installed and configured Hadoop services HDFS, Yarn, MapReduce, Spark, HBase, Oozie, Hive, Sqoop, Flume, Kafka and Sentry.
• Configured Fair schedulers in cluster, created resource pools, and dynamic resource allocation of resources during regular monitoring of resource intensive jobs
• Involved in implementing High Availability and automatic failover infrastructure to overcome single point of failure for Name node utilizing zookeeper services
• Day to day responsibilities includes solving hadoop developer issues and providing instant solution to reduce the impact and documenting the same and preventing future issues.
• Interacting with Cloudera support and log the issues in Cloudera portal and fixing them as per the recommendations.
• Experienced in upgrades, patching, Rolling Upgradesactivities without any data loss and with proper backup plans.
• Integrated external components like Tibco and Tableau with Hadoop using Hive server2.
• Implemented HDFS snapshot feature, migrated data across clusters using DISTCP.
• Performed both major and minor upgrades to the existing Cloudera Hadoop cluster.
• Integrated Hadoop with Active Directory and enabled Kerberos for Authentication.
• Build a new sandbox cluster for the testing purpose and move data from secure cluster to insecure sandbox cluster by usinga tool DISTCP (distributed copy).
• Installed Kafka cluster with separate nodes for brokers.
• Performed Kafka operations on regular basis.
• Expertise in Performance tuning and optimized Hadoop clusters to achieve high performance.
• Implemented schedulers on the Resource Manager to share the resources of the cluster.
• Monitoring Hadoop Clusters using Cloudera Manager and 24x7 on call support.
• Expertise in implementation and designing of disaster recovery plan for Hadoop Cluster.
• Extensive hands on experience in Hadoop file system commands for file handling operations.
• Worked on Providing User support and application support on Hadoop Infrastructure.
• Prepared System Design document with all functional implementations.
• Worked with SQOOP import and export functionalities to handle large data set transfer betweentraditional databases and HDFS.

Environment: Hadoop Hdfs, Mapreduce, Hive, Pig, Oozie, Sqoop, Cloudera Manager, Storm, AWS S3, Ec2, IAM, Zookeeper, spark

Confidential, Valley Forge, PA

Hadoop Admin

Responsibilities:

• Installed and configured Cloudera Manager for easy management of existing Hadoop cluster.
• Conducting RCA to find out data issues and resolve production problems.
• Worked on Hive optimization techniques to improve the performance of long running jobs.
• Experienced in managing and reviewing Hadoop log files
• Worked with Sqoop in Importing and exporting data from different RDMSinto HDFS and Hive.
• Worked on setting up HA for major production cluster and designed automatic failovercontrol using zookeeper and quorum journal nodes.
• Experience on HBase High availability and manually tested using failover tests.
• Create queues and allocated the clusters resources to provide the priority for jobs.
• Experience in upgrading the cluster to newer versions of CDH 5.8.2 and CM 5.9.1
• Provisioning, installing, configuring, monitoring, and maintaining HDFS, Yarn, HBase, Flume, Sqoop, Oozie, Pig, Hive, Falcon, Smartsense, Storm, Kafka and Spark.
• Implemented Kerberos for authenticating all the services in Hadoop Cluster.
• Continuous monitoring and managing the Hadoop cluster through Cloudera manager.
• Created user accounts and given users the access to the Hadoop cluster.

Environment: Hadoop Hdfs, MapReduce, Hive, Pig, Flume, Oozie, Sqoop, spark, Cloudera Manager.

Confidential, Westbury, NY

Hadoop Admin

Responsibilities:

• Involved in deploying a Hadoop cluster using Hortonworks Ambari HDP 2.2 integrated with Site scope for monitoring and Alerting.
• Launching and Setup of HADOOP Cluster on physical servers, which includes configuring different components of HADOOP.
• Created a local YUM repository for installing and updating packages.
• Developed data pipelines that ingests data from multiple data sources and process them.
• Expertise in Using Sqoop to connect to the ORACLE, MySQL, SQL Server, TERADATA and move the pivoted data to Hive tables or HBase tables.
• Implemented Kerberos authentication-KDC server setup, creating realm /domain, managing principles, generating key tab file for each service and managing key tab using key tab tools.
• Configured Knox for perimeter security and Ranger for granular access in the cluster.
• Configured and installed several Hadoop clusters in both on-premises and AWS cloud for POCs.
• Configured and deployed hive metastore using MySQL and thrift server.
• Involved in creating Hive tables, and loading and analyzing data using hive queries
• Extensively used Sqoop to move the data from relational databases to HDFS.
• Used Flume to move the data from web logs onto HDFS.
• Used Pig to apply transformations, validations, cleaning and deduplication of data from sources.
• Actively monitored the Hadoop Cluster with Hortonworks distribution with HDP 2.4.
• Performed various configurations, which includes, networking and Iptables, resolving hostnames, user accounts and file permissions, http, ftp, SSH keyless login.
• Worked on performing minor upgrade from HDP 2.2.2 to HDP 2.2.4
• Upgraded the Hadoop cluster from HDP 2.2 to HDP 2.4 and HDP 2.4 to HDP 2.5
• Integrated BI tool Tableau to run visualizations over the data.
• Solving hardware related Issues Ticket assessment on daily basis.
• Automate administration tasks using scripts and Job Scheduling using CRON.
• Provided 24 x 7 on call support as part of a scheduled rotation with other team members

Environment: HADOOP HDFS, MAPREDUCE, HIVE, PIG, OOZIE, SQOOP, AMBARI, STORM, AWS S3, EC2, IDENTITY ACCESS MANGEMENT, ZOOKEEPER, NIFI

Confidential

DBA

Roles & Responsibilities:

• Maintained SQL Server DB; ensured accuracy of information and automated numerous functions.
• Fine tuning of database objects and server to ensure efficient data retrieval.
• Monitor and optimize system performance using SQL Profiler and DB Engine Tuning Advisor.
• Designed and implemented incremental and full back up policies and procedures.
• Created and implemented database design solutions in collaboration with programming team.
• Developed user defined functions and triggers to implement the requirements of the business.
• Performed database logical and physical design, maintenance, tuning, archiving, backups,replication, recovery, software upgrades, capacity planning and optimization for SQL Server database.
• Database consistency checks using DBCC utilities, Performance Baselining, Performance Tuning
• Production level support for onsite and offshore Clients.
• SQL Server Performance Dashboard Reports for Monitoring.
• Database Security management

Confidential

SQL Developer

Roles & Responsibilities:

• Developed stored procedures, functions and database triggers. Maintained referential integrity and implemented complex business logic.
• Involved in installation and configuration of SQL server 2005 with latest service packs.
• Created and executed SSIS packages to populate data from the various data sources.
• Created SSIS packages using SSIS designer for export heterogeneous data from OLEDB Source (Oracle), Excel spreadsheet to SQL Server 2005.
• Migrated DTS packages to SSIS packages and modified those packages.
• Designed ETL packages dealing with different data source and loaded the data into target data sources by performing different kinds of transformations using SSIS.
• Experience in creating multiple reports (SSRS) in Drill mode using tables, crosstabs, and charts. Design, deployment and maintenance of various SSRS in SQL Server 2005.
• Designed and implemented parameterized and cascading parameterized reports using SSRS.
• Managed the security of servers, creating the new logins and users, changing roles of users.
• Involved in developing logical and physical model of database using Erwin.