SUMMARY:

• Over 8+ years of IT experience including 3.5 years of experience with Hadoop Ecosystem in installation and configuration of different Hadoop eco - system components in the existing cluster.
• Experience in Hadoop Administration (HDFS, MAP REDUCE, HIVE, PIG, SQOOP, FLUME, OOZIE, and HBASE) and NoSQL Administration.
• Experience in installing Hadoop cluster using different distributions of Apache Hadoop, Cloudera and Horton works.
• Extensive experience in installation and configuration of Spark for Testing and Development Environments.
• Experience in implementing Data Warehousing/ETL solutions for different Domains like financial, telecom, and insurance.
• Experience in configuring Zookeeper to provide Cluster coordination services.
• Experience in benchmarking, performing backup and recovery of Name node metadata, and data residing in the cluster.
• Experience in understanding the security requirements for Hadoop and integrating with Kerberos authentication infrastructure- KDC server setup, creating and managing the realm domain.
• Experience with system integration, capacity planning, performance tuning, system monitoring, system security, operating system hardening and load balancing.
• Installed, configured and maintained HBASE.
• Installation, patching, upgrading, tuning, configuring and troubleshooting Linux based operating systems RedHat and Centos and virtualization in a large set of servers.
• Implemented rack aware topology on the hadoop cluster.
• Defining job flows in Hadoop environment using tools like Oozie for data scrubbing and processing.
• Using the Crontab in UNIX and Scheduler for Automatic Tasks.
• Familiar in commissioning and decommissioning of nodes on Hadoop Cluster
• Successfully loaded files to Hive and HDFS from Oracle, SQL Server, MySQL, and Teradata using Sqoop.
• Loading logs from multiple sources directly into HDFS using tools like Flume.
• Worked on Oracle, Teradata and Vertica database systems with Good experience in UNIX Shell scripting.
• Good scripting knowledge in Bash shell scripting.
• Usage of different Talend Hadoop Component likes Hive, Pig and Spark.
• Creating and maintenance of OS users and privileges, maintaining permissions for directories and files.
• Highly motivated and adaptive with the ability to grasp things quickly and possess excellent interpersonal, technical and communication skills.

TECHNICAL SKILLS:

Hadoop Ecosystem Components: Hadoop 2.x, HDFS, Map Reduce, Pig, Hive, Flume, Oozie,Zookeeper, Cloudera Manager Enterprise, HBase and Sqoop.

Scripting Languages: Shell scripting.

Hadoop Distributors: Cloudera Manager, Horton works HDP.

Databases: SQL server, Mongodb, Oracle, MYSQL &NoSQL -CassandraTeradata SQL Assistance, BI Tool SSIS, SSRS.:

Operating Systems: Windows & Linux (CentOS 5, 6 RHEL 6), CentOS, Red HatLinux.

PROFESSIONAL EXPERIENCE:

Confidential, Atlanta, GA

Sr. Hadoop Administrator

Responsibilities:

• Currently working as admin in Horton works (HDP 2.2.4.2) distribution for 4 clusters ranges from POC to PROD.
• Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage &review log files.
• Day to day responsibilities includes solving developer issues, deployments moving code from one environment to other environment, providing access to new users and providing instant solutions to reduce the impact and documenting the same and preventing future issues.
• Experienced in Ambari-alerts configuration for various components and managing the alerts.
• Experienced on adding/installation of new components and removal of them through Ambari.
• Monitoring systems and services through Ambari dashboard to make the clusters available for the business.
• Experienced in Setting up the project and volume setups for the new projects.
• Experienced in managing and reviewing log files.
• Hand on experience on cluster up gradation and patch upgrade without any data loss and with proper backup plans.
• Working experience on maintaining MySQL databases creation and setting up the users and maintain the backup of cluster metadata databases with corn jobs.
• Architecture design and implementation of deployment, configuration management, backup, and disaster recovery systems and procedures.
• Changing the configurations based on the requirements of the users for the better performance of the jobs.
• Provided security and authentication with ranger where ranger admin provides administration and user sync adds the new users to the cluster.
• Good troubleshooting skills on Hue, which provides GUI for developer's/business users for day to day activities.
• Developed Map Reduce programs to cleanse the data in HDFS obtained from heterogeneous data sources to make it suitable for ingestion into Hive schema for analysis.
• Implemented complex Map Reduce programs to perform joins on the Map side using distributed cache
• Setup flume for different sources to bring the log messages from outside to Hadoop HDFS.
• Implemented Name Node HA in all environments to provide high availability of clusters.
• Capacity scheduler implementation in all environments to provide resources based on the allocation.
• Create queues and allocated the clusters resources to provide the priority for jobs.
• Involved in snapshots and mirroring to maintain the backup of cluster data and even remotely.
• Implementing the SFTP for the projects to transfer data from External servers to servers.
• Setting up MySQL master and slave replications and helping business applications to maintain their data in MySQL Servers.
• Helping the users in production deployments throughout the process.
• Experienced in production support which involves solving the user incidents varies from sev1 to sev5.
• Managed and reviewed Log files as a part of administration for troubleshooting purposes. Communicate and escalate issues appropriately.
• As an admin followed standard Back up policies to make sure the high availability of cluster.
• Involved in Analyzing system failures, identifying root causes, and recommended course of actions.
• Documented the systems processes and procedures for future references.
• Worked with systems engineering team to plan and deploy new environments and expand existing clusters.
• Monitored multiple clusters environments using AMBRI Alerts, Metrics and Nagios.

Environment: Hadoop HDFS, Map reduce, Hive, Pig, Flume, Oozie, Sqoop, Eclipse, Horton works,Ambari.

Hadoop Administrator

Responsibilities:

• Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage &review log files.
• Day to day responsibilities includes solving developer issues, deployments moving code from one environment to other environment, providing access to new users and providing instant solutions to reduce the impact and documenting the same and preventing future issues.
• Experienced on adding/installation of new components and removal of them through Ambari.
• Implemented and Configured High Availability Hadoop Cluster(Quorum Based)
• Installed and Configured Hadoop monitoring and Administrating tools: Nagios and Ganglia.
• Back up of data from active cluster to a backup cluster using distcp.
• Periodically reviewed Hadoop related logs and fixing errors and preventing errors by analyzing the warnings.
• Hands on experience working on Hadoop ecosystem components like Hadoop Map Reduce, HDFS, Zookeeper, Oozie, Hive, Sqoop, Pig, Flume.
• Experience in configuring Zookeeper to coordinate the servers in clusters to maintain the data consistency.
• Experience in using Flume to stream data into HDFS - from various sources.
• Used Oozie workflow engine to manage interdependent Hadoop jobs and to automate several types of Hadoop jobs such as Java map-reduce, Hive and Sqoop as well as system specific jobs.
• Installed Oozie workflow engine to run multiple Hive and pig jobs.
• Worked on analyzing Data with HIVE and PIG +9.
• Helped in setting up Rack topology in the cluster.
• Upgraded the Hadoop cluster from CDH3 to CDH4.
• Deployed a Hadoop cluster using CDH3 integrated with Nagios and Ganglia.
• Implemented Fair scheduler on the job tracker to allocate the fair amount of resources to small jobs.
• Implemented automatic failover zookeeper and zookeeper failover controller.
• Deployed Network file system for Name Node Metadata backup.
• Performed cluster back using DISTCP, Cloudera manager BDR and parallel ingestion.
• Performed both major and minor upgrades to the existing cluster and also rolling back to the previous version.
• Designed the cluster so that only one secondary name node daemon could be run at any given time.
• Implemented commissioning and decommissioning of data nodes, killing the unresponsive task tracker and dealing with blacklisted task trackers.
• Dumped the data from HDFS to MYSQL database and vice-versa using SQOOP.

Environment: Hadoop, Map reduce, HDFS, Hive, Pig, Java, SQL, Cloudera Manager, Sqoop, Flume,Oozie, Zookeeper, MySQL, Java, Eclipse, Linux

Hadoop Administrator

Responsibilities:

• Installed, Configured and Maintained the Hadoop cluster for application development and Hadoop ecosystem components like Hive, Pig, HBase, Zookeeper and Sqoop.
• In depth understanding of Hadoop Architecture and various components such as HDFS, Name Node, Data Node, Resource Manager, Node Manager and YARN / Map Reduce programming paradigm.
• Monitoring Hadoop Cluster through Cloudera Manager and Implementing alerts based on Error messages. Providing reports to management on Cluster Usage Metrics and Charge Back customers on their Usage.
• Extensively worked on commissioning and decommissioning of cluster nodes, replacing failed disks, file system integrity checks and maintaining cluster data replication.
• Very good understanding and knowledge of assigning number of mappers and reducers to Map reduce cluster.
• Setting up HDFS Quotas to enforce the fair share of computing resources.
• Strong Knowledge in Configuring and maintaining YARN Schedulers (Fair, and Capacity).
• Wrote the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
• Experience in setting up HBase cluster which includes master and region server configuration, High availability configuration, performance tuning and administration.
• Created user accounts and given users the access to the Hadoop cluster.
• Involved in loading data from UNIX file system to HDFS.
• Worked on ETL process and handled importing data from various data sources, performed transformations.
• Coordinate with QA team during testing phase.
• Provide application support to production support team.

Environment: Cloudera 5.3, HDFS, Hive, Sqoop, Zookeeper and HBase, Windows 2000/2003 Unix Linux Java, HDFS Map Reduce, Pig Hive HBase Flume Sqoop, Shell Scripting.

Linux Administrator

Responsibilities:

• Gathering requirements from customers and business partners and design, implement and provide solutions in building the environment.
• Creating physical volumes, volume groups, and logical volumes.
• Monitored the system performance and network related issues through vmstat, iostat, grapher tool.
• Installing and configuring Apache and supporting them on Linux production servers.
• Configured backup and restoration of data from Dump and Target location using Shell Scripts.
• Worked with heterogeneous Client & Server management.
• Troubleshooting of Solaris 8/9/10 and Red-Hat 3/4 on Sun Enterprise Servers and AIX on IBM Blade Servers.
• Worked on package installations, patch up gradations and other server management techniques.
• Recovered data from server after a ROOT file system crash.
• Administered the Network of 400 UNIX Users using Shell Scripts.
• Maintaining relations with project managers, DBA's, Developers, Application support teams and Operational support teams to facilitate effective project deployment.
• Handled Day-to-Day operation of all servers running Production jobs, Backups, Restores and Report generations.
• Planned for storage and backup including analyzing the disk space requirement and backup without affecting the device performance.
• Used heterogeneous backup software for Windows and UNIX to backup and retrieve file systems without loss of data.

Environment: LINUX, FTP, Shell, UNIX, VMware, NFS, TCP/IP, Oracle Red Hat Linux.

Linux Administrator

Responsibilities:

• Installation and configuration of Linux for new build environment.
• Day-to- day - user access, permissions, Installing and Maintaining Linux Servers.
• Created volume groups logical volumes and partitions on the Linux servers and mounted file systems and created partitions.
• Installed Cent OS using Pre-Execution environment boot and Kick start method on multiple servers, remote installation of Linux using PXE boot.
• Monitoring the System activity, Performance, Resource utilization.
• Develop and optimize physical design of MySQL database systems.
• Automate administration tasks through use of scripting and Job Scheduling using CRON.
• Deep understanding of monitoring and troubleshooting mission critical Linux machines.
• Created Virtual server on Citrix Xen Server based host and installed operating system on Guest Servers.
• Responsible for maintenance Raid-Groups, LUN Assignments as per agreed design documents. Performed all System administration tasks like Cron jobs, installing packages, and patches.
• Extensive use of LVM, creating Volume Groups, Logical volumes.
• Performed RPM and YUM package installations, patch and other server management.
• Performed scheduled backup and necessary restoration.
• Performed configuration and troubleshooting of services like NFS, NIS, NIS+, DHCP, FTP, LDAP, Apache Web servers.
• Managed critical bundles and patches on the production servers after successfully navigating through the testing phase in the test environments.
• Managing Disk File Systems, Server Performance, Users Creation and Granting file access Permissions and RAID configurations.
• Updating YUM Repository and Red hat Package Manager (RPM).
• Configured Domain Name System (DNS) for hostname to IP resolution.
• Preparation of operational testing scripts for Log check, Backup and recovery and Failover.
• Troubleshooting and fixing the issues at User level, System level and Network level by using various tools and utilities. Schedule backup jobs by implementing cron job schedule during non-business hour.

Environment: YUM, RAID, MYSQL 5.1.4, PHP, SHELL SCRIPT, MYSQL, WORKBENCH, LINUX 5.0, 5.1, YUM, RAID.

Linux Administrator

Responsibilities:

• Creating, managing users and user groups.
• Installed numerous operating systems (i.e. Windows NT / 2000 / XP / Vista & Linux )
• Produced and applied script for automation of routine tasks like 8-hour net disconnection and daily log backup on Linux servers.
• Hands on experience in the installation, configuration, backup/restore using VERITAS net backup (NBU) and Tivoli Storage Management (TSM).
• Strong working knowledge of Routers and Switches, Subnetting, VLAN, TCP / IP, VPN, OSI model, Firewall (checkpoint), VOIP and Sarbanes Oxley compliance (SOX).
• Building Universe, retrieving data using personal data files, Freehand SQL methods and creating complex Adhoc reports.
• Responsible for maintaining the integrity and security of the enterprise UNIX (Linux /Solaris) servers and systems.
• Maintained a disaster recovery plan. Creating backup capabilities adequate for the recovery of data and understands concepts and processes of replication for disaster recovery.
• Maintained DNS, NFS, and DHCP, printing, mail, web, and FTP services for the enterprise.
• Managed UNIX account maintenance including additions, changes, and removals.
• User administration for all the NIS users.
• Forecast storage needs and worked with the site management to determine future disk requirements.
• Worked with the application teams and other IT department personnel to coordinate system software changes and support application changes
• Working knowledge on the TCP/IP protocols RSH, SSH, RCP, SCP.
• Operating system support - Windows 2003/2008, RHEL4/5/6, SLES 9.
• Installed and Configured Virtual Machines (VM), VM tools on ESX boxes from the GUI and console.
• Building/upgrading servers (DNS, DHCP, NIS, HTTPD, Cups) and general-purpose servers using PXE-boot with kick-start.
• Installing and upgrading software on Linux (rpm/yum) and Windows Systems using puppet
• Responsible for complete user management which includes creating, modifying, closing user accounts on Linux/Unix and Windows Systems (Active Directory)
• Setting up NIS domains on Red hat, CentOS and Solaris systems, which involves configuring servers and client roles.
• Worked on Creating Local zones and Branded zones on Solaris Containers global zone and migration of the applications on to these zones from the physical servers.
• Created hierarchy and conditions at the design level.
• Applied recommended OS Patches, Kernel upgrades using Solaris Live Upgrade on Production and development servers.
• Migrating physical devices to virtual devices and also virtual devices to physical devices. The task includes upgrades of operating system, hardware, and applications through the process.
• Demonstrated/Trained the staff in Understanding the Project Front end application to understand the business.
• Co-ordinate with development team by assigning tasks and deliverables and resolving high priority issues
• Designed and implemented a Virtual Environment utilizing Citrix Infrastructure and XenServer.
• Utilized the SAN as the proper data store for the virtual environment.
• Deployed Citrix XenServer on IBM Blade environment communicating via Brocade 200E using EMC AX4-5 SAN

Environment: Linux, Solaris, TCP/IP, Cent OS, Citrix XenServer, Tivoli Storage Management, IBM Blade.