SUMMARY

• 12 years of total IT experience in which 4 plus years’ experience in Hadoop administration.
• Experience on Hadoop Administration, responsibilities include software installation, configuration, software upgrades, backup and recovery, cluster setup, cluster performance and monitoring on daily basis, maintaining cluster up and run on healthy.
• Experience in architecting, designing, installation, configuration and management of Apache HadoopClusters, Hortonworks Data Platform & ClouderaHadoopDistribution.
• Strong Experience in Installation and configuration of Cloudera distribution Hadoop 4.x/5.x and Hortonworks Data Platform 2.2/2.3/2.4.
• Experience in understanding and managing Hadoop Log Files.
• Experience in understanding the security requirements forHadoopand integrating with Kerberos authentication infrastructure - KDC server setup, creating realm /domain, managing
• Experience in commissioning and decommissioning the nodes in Hadoop Cluster.
• Experience in architecting, designing, installation, configuration and management of Apache Hadoop Clusters & Cloudera Hadoop Distribution.
• Experience in Hadoop Shell commands, verifying managing and reviewing Hadoop Log files.
• Experience in Implementing High Availability of Name Node and Hadoop Cluster capacity planning
• Setting up HDFS Quotas to enforce the fair share of computing resources. Rebalance an HDFS Cluster.Successfully loaded files to Hive and HDFS from Oracle, SQL Server, MySQL, and Teradata using SQOOP.
• Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop Map Reduce, HDFS, HBASE, OOZIE, HIVE, SQOOP, PIG and Zoo keeper.
• Excellent understanding / knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce.
• Optimizing performance of HBASE/Hive/Pig jobs.
• Experience in managing the Hadoop infrastructure with HDP and Cloudera Manager
• Extensively worked on the ETL mappings, analysis and documentation of OLAP reports requirements.
• Solid understanding of OLAP concepts and challenges, especially with large data sets.
• Experience on UNIX commands and Shell Scripting.
• Excellent interpersonal, communication, documentation and presentation skills.
• Experience supporting systems with 24X7 availability and monitoring

TECHNICAL SKILLS

Hadoop Ecosystem: HDFS, HBase,Impala, HadoopMapReduce, Zookeeper, Hive, Pig, Sqoop, Flume, Oozie, Cassandra.

Hardware/Operating Systems: Windows NT, Windows2003, 2008 R2 and 2012 server, UNIX, AIX 4.3.x, Solaris9/10, RHEL 5.x, 6.x and CentOS, HP-UX 11.0 and 11i, SAN.

Development Languages: HiveQL, SQL, PL/SQL, C, C++, PHP, Python, Core Java, JavaScript, Shell Script, Perl script, Visual Source Safe, Crystal Reports, Red Gate, Erwin, Visio

PROFESSIONAL EXPERIENCE

Confidential, NYC

Sr. Hadoop Administrator

RESPONSIBILITIES:

• Responsible for Cluster configuration maintenance and troubleshooting and tuning the cluster.
• Secure a deployment and understand Backup and Recovery.
• Involved in creating Hive tables loading with data and writing hive queries which will run internally in map reduce way
• Rack Aware Configuration, Configuring Client Machines Configuring, Monitoring and Management Tools
• File system management and monitoring. HDFS support and maintenance.
• Responsible for building a cluster on HDP 2.3 and HDP2.4
• Collaborating with application teams to install operating system andHadoopupdates, patches, version upgrades when required. Point of Contact for Vendor escalation
• Major Upgrade from HDP 2.2 to HDP 2.4.
• Created POC to store Server Log data into Cassandra to identify System Alert Metrics
• Rack Aware Configuration and AWS working nature
• Good experience in troubleshoot production level issues in the cluster and its functionality.
• Deployed Puppet, Puppet Dashboard, and Puppet DB for configuration management to existing infrastructure.
• Installation and Configuration of other Open Source Software like Pig, Hive, HBASE, Flume and Sqoop
• Changes to the configuration properties of the cluster based on volume of the data being processed and performance of the cluster.
• Working with dev Team to tune Job Knowledge of Writing Hive Jobs.
• Set up and manage HA name node and Name node federation using Apache 2.0 to avoid single point of failures in large clusters.
• Configuring Flume for efficiently collecting, aggregating and moving large amounts of log data from many different sources to HDFS.
• Involving in collecting and aggregating large amounts of streaming data into HDFS using Flume and defined
• Working with data delivery teams to setup newHadoopusers. This job includes setting up Linux users, setting up Kerberos principals and testing MFS, Hive.
• Working with HortonWorks Support Team to Fine tune Cluster
• Built data platforms, pipelines, and storage systems using the Apache Kafka, Apache Storm and search technologies such as Elastic search.
• Responsible for scheduling jobs inHadoopusing FIFO, Fair scheduler and Capacity scheduler
• Set up automated processes to analyze the System andHadooplog files for predefined errors and send alerts to appropriate groups.

ENVIRONMENT: RHEL Linux 5.x/6.x, HDP2.4, Ambari 2.2.1, Hbase 1.1.2, Apache Hadoop 2.7, YARN, HDFS, Hive 1.2, Pig 0.15, Flume 1.5, Sqoop1.4, SPARK1.6, ZK 3.4.6, Kafka 0.9, Tez, 0.7, Ranger 0.5.0, Knox 0.6, Kerberos.

Confidential, Hoboken, NJ

Hadoop Administrator

RESPONSIBILITIES:

• Responsible for architectingHadoopclusters Translation of functional and technical requirements into detailed architecture and design.
• Worked exclusively on Cloudera distribution ofHadoop.
• Installed and configured multi-node fully distributedHadoopcluster of large number of nodes.
• Setting up the machines with Network Control, Static IP, Disabled Firewalls, and Swap memory.
• Installed and configured Cloudera Manager for easy management of existingHadoopcluster.
• Worked on setting up high availability for major production cluster and designed automatic failover control using Zookeeper and Quorum Journal Nodes.
• Implemented Fair scheduler on the job tracker to allocate fair amount of resources to small jobs.
• Performed operating system installation andHadoopversion updates using automation tools.
• Configured Oozie for workflow automation and coordination.
• Implemented rack aware topology on theHadoopcluster.
• Importing and exporting structured data from different relational databases into HDFS and Hive using Sqoop.
• Configured ZooKeeper to implement node coordination in clustering support.
• Configured Flume for efficient collection, aggregation and transformation of huge log data from various sources to HDFS.
• Involved in collecting and aggregating large amounts of streaming data into HDFS using Flume and defined channel selectors to multiplex data into different sinks.
• Working with data delivery teams to setup newHadoopusers. This job includes setting up Linux users, setting up Kerberos principals and testing MFS, and Hive
• Implemented Kerberos Security Authentication protocol for existing cluster.
• Backed up data on regular basis to a remote cluster using distcp.
• Experience in managing backups and version upgrades
• Good experience in troubleshoot production level issues in the cluster and its functionality.
• Regular Commissioning and Decommissioning of nodes depending upon the amount of data.
• Monitored and configured a test cluster on AWS for further testing process and gradual migration.
• Experience in deploying and managing the multi-node development, testing and production.

ENVIRONMENT: CentOS 6.x, RHEL 6.x, CDH4.x, Apache Hadoop 2.2, Hive 0.12, Pig 0.12, Sentry, Hue 3.9.0, Shell Scripting, ZooKeeper 3.4.5, Sqoop 1.4, Crunch 0.11, Flume 1.6, Oozie 4.1.0, YARN.

Confidential, Dublin, OH

Linux System Administrator

RESPONSIBILITIES:

• Installed, configured & handled troubleshooting of Solaris 8/9/10 and Red-Hat 3/4 on Sun Enterprise Servers and AIX on IBM Blade Servers.
• Installed, configured, troubleshoot and maintain Linux Servers and Apache Web server, configuration and maintenance of security and scheduling backups, submitting various types of cron jobs
• Administration, package installation, configuration of Oracle Enterprise Linux 5.x.
• Administration of RHEL, which includes installation, testing, tuning, upgrading and loading patches, troubleshooting both physical and virtual server issues.
• Deployed and implemented NIS, NIS+ and DNS environment and implemented and maintained Proxy Server under Linux; issues regarding Samba Servers and perform hardware and software installs/upgrades.
• Administered the Network of 400 UNIX Users using Shell Scripts.
• Set up the Domain Name System by configuring the DNS primary, secondary and DNS Clients and configured the database files.
• Installing RedHat Linux using kick start and applying security polices for hardening the server based on the company policies.
• RPM and YUM package installations, patch and other server management.
• Installed several security packages like SNORT, tcode, tp-wrappers & ASSET.
• Part of team for Installation and configuration of DB2.
• Tech and non-tech refresh of Linux servers, which includes new hardware, OS, upgrade, application installation, testing.
• Handled Day-to-Day operation of all servers running Production jobs, Backups, Restores and Report generations.
• Backed up and restored data from tape in ufsdump, tar, poi& did format.
• Performed various configurations that include networking and IPTable, resolving hostnames, SSH key less login.
• Configured the Send Mail configuration file, created e-mail ids, created alias database.
• Planned for storage and backup including studying the disk space requirement and backup device performance.
• Implemented security in systems and network for all Sun Enterprise Servers & Desktop Servers.
• Managing systems routine backup, scheduling jobs like disabling and enabling cron jobs, enabling system logging, network logging of servers for maintenance, performance tuning, testing.

ENVIRONMENT: Oracle 9i/10g, RHEL Linux 3.x/4.x, Solaris 8/9, Windows NT, Quest Backup software.

Confidential, Buffalo - NY

System Admin

RESPONSIBILITIES:

• Responding to the system generated action items and resolve them depending on the contract level of the customer.
• Co-coordinating with the customers vendors for any system up gradation and giving the exact procedure to follow up.
• Providing the monthly Management reports/statistics.
• Scheduling the daily/weekly/monthly backups.
• Patching up the system to the latest version as per the recommendations.
• Monitor the health of the servers, Operating system, database and the network.
• Maintenance of Hard disks (Formatting and Setup, Repair from crashes)
• Create and maintain user accounts administering file systems and recognizing file access problems.
• Granting the required access codes to various groups Performance tuning, Maintaining disks through VERITAS Volume Manager, Crontab, Growing/shrinking VxFS file systems
• Taking disks out of VxVM, VERITAS Cluster support.
• Performing patches research, patches installation and packages installation on Sun Solaris
• Fine tuning of servers and configuring networks for optimum performance
• Planning and implementing system upgrades including hardware, operating system and periodical patch upgrades setting up RAID levels on Sun Storage Equipments using VERITAS Volume Manager and Solstice Disk suite on new/existing production/development systems for reliability, fault tolerance, and availability
• Adding/expanding new storage to existing/new systems using VERITAS Volume Manager
• Writing shell scripts as per requirements
• Creating, Scheduling and Managing Cron jobs
• Handling Support cases based on priority and SLA.

ENVIRONMENT: Sun Solaris 8, Sun Enterprise servers E4500, E3500, Sun Fire V480, sun Ultra Sparc 5/10, Sun Blade servers, Oracle 8i/9i, NFS, BMC patrol and HP OpenView.