SUMMARY

• Over 8 years of IT experience including 2.5 years of experience wif Hadoop Ecosystem in installation and configuration of different Hadoop eco - system components in teh existing cluster.
• Experience in Hadoop Administration (HDFS, MAP REDUCE, HIVE, PIG, SQOOP, FLUME AND OOZIE,HBASE) NoSQL Administration
• Experience in deploying Hadoop cluster on Public and Private Cloud Environment like Amazon AWS.
• Setting up automated 24x7 monitoring and escalation infrastructure for Hadoop cluster using Nagios and Ganglia.
• Experience in installing Hadoop cluster using different distributions of Apache Hadoop, Clouderaand Hortonworks.
• Good Experience in understanding teh client's Big Data business requirements and transform it into Hadoop centric technologies.
• Analyzing teh clients existing Hadoop infrastructure and understand teh performance bottlenecks and provide teh performance tuning accordingly.
• Installed, Configured and maintained HBASE
• Worked wif Sqoop in Importing and exporting data from different databases like MySQL,Oracle into HDFS and Hive.
• Defining job flows in Hadoop environment using tools like Oozie for data scrubbing and processing.
• Experience in configuring Zookeeper to provide Cluster coordination services.
• Loading logs from multiple sources directly into HDFS using tools like Flume.
• Good experience in performing minor and major upgrades.
• Experience in benchmarking, performing backup and recovery of Name node metadata and data residing in teh cluster.
• Familiar in commissioning and decommissioning of nodes on Hadoop Cluster.
• Adept at configuring Name Node High Availability.
• Worked on Disaster Management wif Hadoop Cluster.
• Well experienced in building servers like DHCP, PXE wif kick-start, DNS and NFS and used them in building infrastructure in a Linux Environment.
• Experienced in Linux Administration tasks like IP Management (IP Addressing, Subnetting Ethernet Bonding and Static IP).
• Strong noledge on Hadoop HDFS architecture and Map-Reduce framework.
• Experience in deploying and managing teh multi-node development, testing and production
• Experience in understanding teh security requirements for Hadoop and integrating wif Kerberos autantication infrastructure- KDC server setup, creating realm /domain, managing
• Principles, generating key tab file for each and every service and managing key tab using key tab tools.
• Worked on setting up Name Node high availability for major production cluster and designed Automatic failover control using zookeeper and quorum journal nodes. Authorized to work in teh US for any employer.

TECHNICAL SKILLS:

Hadoop Ecosystems: HDFS, Mapreduce-YARN/MR1, Hbase, Hive, Pig, Sqoop, Oozie, Flume

Databases Oracle: Tera Data, DB2, MySQL sever 2005 and PostgreSql

Platforms: Windows, Ubuntu, Centos, RHEL, SolarisLanguages Bash shell scripting, SQL

PROFESSIONAL EXPERIENCE

Hadoop Admin/Consultant

Confidential, New York, NY.

Responsibilities:

• Responsible for architecting Hadoop clusters Translation of functional and technical requirements into detailed architecture and design.
• Installed and configured multi-nodes fully distributed Hadoop cluster of large number of nodes.
• Provided Hadoop, OS, Hardware optimizations.
• Setting up teh machines wif Network Control, Static IP, Disabled Firewalls, Swap memory.
• Installed and configured Hadoop ecosystem components like MapReduce, Hive, Pig, Sqoop, HBase, ZooKeeper and Oozie.
• Involved in testing HDFS, Hive, Pig and MapReduce access for teh new users.
• Cluster maintenance as well as creation and removal of nodes using Cloudera and Hortonworks Manager Enterprise.
• Worked on setting up high availability for major production cluster and designed automatic failover control using zookeeper and quorum journal nodes.
• Implemented Fair scheduler on teh job tracker to allocate fair amount of resources to small jobs.
• Performed operating system installation, Hadoop version updates using automation tools.
• Configured Oozie for workflow automation and coordination.
• Implemented rack aware topology on teh Hadoop cluster.
• Importing and exporting structured data from different relational databases into HDFS and Hive using Sqoop.
• Configured ZooKeeper to implement node coordination, in clustering support.
• Configured Flume for efficiently collecting, aggregating and moving large amounts of log data from many different sources to HDFS.
• Involved in collecting and aggregating large amounts of streaming data into HDFS using Flume and defined channel selectors to multiplex data into different sinks.
• Worked on developing scripts for performing benchmarking wif Terasort/Teragen.
• Implemented Kerberos Security Autantication protocol for existing cluster.
• Good experience in troubleshoot production level issues in teh cluster and its functionality.
• Backed up data on regular basis to a remote cluster using DistCp.
• Regular Commissioning and Decommissioning of nodes depending upon teh amount of data.
• Monitored and configured a test cluster on amazon web services for further testing process and gradual migration.

Environment: Architecting Hadoop clusters,MapReduce, Hive, Pig, Sqoop, HBase, ZooKeeper and Oozie, HDFS, Hive, Pig and MapReduce,Cloudera and Hortonworks Manager Enterprise, zookeeper and quorum journal nodes, Terasort/Teragen.

Hadoop Admin/Consultant

Confidential, NY.

Responsibilities:

• Installed HA Name node, Yarn (Resource Manager, Node manager, Application master),Data node.
• Installed and Configured HDP2.2
• Responsible for implementation and ongoing administration of Hadoop infrastructure.
• Monitored already configured cluster of 40 nodes.
• Installed and Configured Hadoop platform cluster of 70 nodes
• Installed and configured Hadoop components Hdfs, Hive, HBase.
• Communicating wif teh development teams and attending daily meetings.
• Addressing and Troubleshooting issues on a daily basis.
• Launched R-statistical tool for statistical computing and Graphics.
• Working wif data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos TEMPprincipals and testing HDFS, Hive.
• Cluster maintenance as well as creation and removal of nodes.
• Monitor Hadoop cluster connectivity and security
• Manage and review Hadoop log files.
• File system management and monitoring.
• HDFS support and maintenance.
• Diligently teaming wif teh infrastructure, network, database, application and business intelligence teams to guarantee high data quality and availability.

Environment: HA Name node,Yarn,Data node,HDP2.2,Hdfs,Hive,HBase,R-statistical,Hdfs

Hadoop Admin/Consultant

Confidential, CT

Responsibilities:

• Handle teh installation and configuration of a Hadoop cluster.
• Build and maintain scalable data using teh Hadoop ecosystem and other open source components like Hive and HBase.
• Monitor teh data streaming between web sources and HDFS.
• Close monitoring and analysis of teh Map Reduce job executions on cluster at task level.
• Inputs to development regarding teh efficient utilization of resources like memory and CPU utilization based on teh running statistics of Map and Reduce tasks.
• Changes to teh configuration properties of teh cluster based on volume of teh data being processed and performance of teh cluster.
• Setting up Identity, Autantication and Authorization.
• Maintaining Cluster in order to remain healthy and in optimal working condition.
• Handle teh upgrades and Patch updates.
• Set up automated processes to analyze teh System and Hadoop log files for predefined errors and send alerts to appropriate groups.
• Inputs to development regarding teh efficient utilization of resources like memory and CPU utilization.
• Based on teh running statistics of Map and Reduce tasks.
• Balancing HDFS manually to decrease network utilization and increase job performance.
• Commission and decommission teh Data nodes from cluster in case of problems.
• Set up automated processes to archive/clean teh unwanted data on teh cluster, in particular on Name node and Secondary name node.
• Discussions wif other technical teams on regular basis regarding upgrades, Process changes, any special processing and feedback.

Environment:Hadoop, MapReduce, Hive, HDFS, PIG, Sqoop, Oozie, Cloudera, HBase, ZooKeeper, CDH3, Oracle, NoSQL and Unix/Linux.

SQL DBA Performance Tester

Confidential

Responsibilities:

• Administration of RHEL, which includes installation, testing, tuning, upgrading and loading patches, troubleshooting both physical and virtual server issues.
• Creating, cloning Linux Virtual Machines. Administration, package installation, configuration of Oracle Enterprise Linux 5.x.
• Installing Red Hat Linux using kick start and applying security polices for hardening teh server based on teh company policies.
• RPM and YUM package installations, patch and other server management.
• Managing systems routine backup, scheduling jobs like disabling and enabling Cron jobs, enabling system logging, network logging of servers for maintenance, performance tuning, testing.
• Tech and non-tech refresh of Linux servers, which includes new hardware, OS, upgrade, application installation, testing.
• Set up user and group login ID's, printing parameters, network configuration, password, resolving permissions issues, and user and group quota.
• Creating physical volumes, volume groups, and logical volumes.
• Gathering requirements from customers and business partners and design, implement and provide solutions in building teh environment.
• Installing and configuring Apache and supporting them on Linux production servers.

Environment:Oracle Shell, Perl, PL/SQL, DNS, TCP/IP, Apache Tomcat, XML, HTML and UNIX/Linux