PROFESSIONAL SUMMARY:

• Around 8 years of extensive experience including 2+ years of Hadoop Administration and 5 years of Linux/Unix Administration
• Experienced Installation/Configuration/Managing/Upgrading the Hortonworks Hadoop Cluster on RedHat Linux servers in multiple environments
• Responsible for upgrading Hortonworks Hadoop HDP2.2 and Mapreduce 2.0 with YARN in Multi Clustered Node environment
• Good knowledge on Cloudera Distributes Hadoop(CDH) Cluster
• Experience upgrading the clusters using Rolling Upgrade(Ambari) and Manual Upgrade(Command line)
• Performed major upgrade in production environment from HDP 1.3 to HDP 2.2
• Implemented Kerberos authentication integrated with enterprise Active Directory, creating realm domain, managing principles, generating key tab files for each and every service and managing key tab using key tab tools
• Involved in the new projects on boarding into the cluster and setting required space quota and permissions for the users to restrict access to respective projects/use cases
• Proactively working with Hortonworks Support team for resolving any cluster related issues
• Debugging failed MapReduce jobs and finding the root cause and providing the solution
• Supporting development team for their questions on Hadoop cluster environment and resolving and helping in connecting to the Hadoop cluster
• Experience on Hadoop cluster maintenance including data and metadata backups, file system checks, commissioning and decommissioning nodes
• As an admin followed standard Back up policies to make sure the high availability of cluster
• Monitor multiple Hadoop clusters environments using Ganglia and Nagios
• Monitor workload, job performance and capacity planning using Ambari
• Install and configure Hortonworks on single node clusters for POCs
• Configured Name node High Availability and Resource Manager High Availability in HDP 2.3.
• Expertise in Cluster Upgrade experience(HDP 2.2.4.2, Ambari 2.0 to HDP 2.3, Ambari 2.1)
• Configured Kerberos through Ambari admin panel
• Created Jenkins server to deploy the code from GitHub onto Hadoop cluster
• Involved in implementing security on Hortonworks Hadoop Cluster with Ranger and Apache Knox
• Created HIVE databases and granted appropriate permissions through Ranger policies
• Experience in working with Ranger in enabling metadata management, governance and audit
• Configured NiFi in Kerberized Cluster
• Implemented search on HDFS using operational metadata stored in Elastic Search/Solr
• Implemented ELK (Elastic Search, Log stash, Kibana) stack to collect and analyze the logs
• Working with Linux Admin for quarterly OS and other security patching and any hardware issues
• Preparing the documentation and run books for supporting/upgrading the cluster
• Monitoring the health of the cluster and making sure the cluster is up, running and healthy
• Hands - on programming experience in various technologies like Shell Scripting, Perl and Python
• Having Working Knowledge on Sqoop and Flume for Data Processing
• Loading the data from the different Data sources like Teradata, MySQL and Oracle into HDFS using Sqoop and load into partitioned Hive tables
• Excellent knowledge of in NOSQL databases like HBase,MongoDB
• Formulated procedures for installation of Hadoop patches, updates and version upgrades and automated processes for troubleshooting, resolution and tuning of Hadoop clusters
• Working with other team members in supporting data ingestion into the cluster
• Involved in log file management where the logs greater than 7 days old were removed from log folder and loaded into HDFS and stored for 3 months
• Managing users and application quota on the cluster
• Installing apache HTTP for hosting internal repository for all the packages
• Installing other third party tools based on the user requirements
• Installed Analytics tool R on Linux servers and debugged connectivity issues from R to Hadoop cluster
• Familiar with data architecture including data ingestion pipeline design, Hadoop information architecture, data modelling and data mining, machine learning and advanced data processing. Experience optimizing ETL workflows
• Supported Web Sphere Application Server WPS, IBM HTTP/ Apache Web Servers in Linux environment for various projects
• Created reports for the BI team using Sqoop to export data into HDFS and Hive
• Supported geographically diverse customers and teams in a 24/7 environments
• Team player with strong analytical, technical negotiation and client relationship management skills
• Developed Oozie workflows and sub workflows to orchestrate the Sqoop scripts, pig scripts, hive queries and the Oozie workflows are scheduled through Autosys
• Conducted detailed analysis of system and application architecture components as per functional requirements
• Ability to work effectively in cross-functional team environments and experience of providing training to business user

TECHNICAL SKILLS:

Operating Systems: Windows, Red hat Linux, IBM AIX, and Mac

Big Data Technologies: Hortonworks Hadoop Cluster, Map-Reduce, HDFS, Sqoop, Pig, Hive, HBase, Zookeeper,Ranger, Spark, Ambari, YARN, Flume, Oozie, Cassandra, Hcatalog, Mahout, Avro, Storm

Hadoop Cluster: Hortonworks, Cloudera

Frameworks: Hadoop MapReduce, Elastic Search, MVC, Struts 2.x/1.x, Hibernate, Springs

JEE Technologies: JSP, Servlets, JDBC, EJB, spring

Programming Languages: Core Java, J2EE, SQL, UNIX Shell Scripting, Perl, Java Script, Python

IDE’s: Eclipse, Net beans

Web Technologies: JSP, JDBC, ODBC, HTML, CSS3, XML, JavaScript, Servlets, Java Beans, Junit

Web/Application servers: Tomcat 7/8, Web Logic Server, Web Sphere, JBoss, Apache

Databases: Oracle 11G, MySQL, Teradata, postgress, MongoDB, Cassandra

Version Control: SVN, GitHub

Tools: ATOM, Eclipse3.5, R, Ganglia, Nagios, Puppet, Kerberos, Remedy, JIRA, Tableau, Active Directory, Jenkins

PROFESSIONAL EXPERIENCE:

Confidential - Tempe, AZ

Hadoop Administrator

Environment: - Hortonworks, MySQL, R, Postgress, Shell Scripting,Elastic Search, Java, Python, RedHat Linux, Autosys, Tableau, JDK1.8, Teradata, Oracle, Nagios, Ganglia

Responsibilities:

• Installing/Configuring/Managing Hortonworks Hadoop cluster in multiple environments
• Integrated Hadoop with Active Directory and enabled Kerberos for Authentication
• Configured NiFi in Kerberized Cluster
• Created roles based on the project requirements to restrict the access for users using Ranger
• Implementation of Ranger, Ranger plug-ins and Knox security tools
• Experience in Ranger, Knox configuration to provide the security for Hadoop services (hive, base, hdfs etc.)
• Worked with Ambari Rest API and built custom scripts to automate the process
• Worked with Ranger, Knox configuration to provide centralized security to Hadoop services
• Coordinated with technical team for production deployment of software applications for maintenance
• Implemented search on HDFS using operational metadata stored in Elastic Search/Solr
• Implemented ELK (Elastic Search, Log stash, Kibana) stack to collect and analyze the logs
• Planning for cluster capacity and submitting the requests for respective hardware and software
• Designed, configured and managed the backup and disaster recovery for HDFS data
• Commissioned Data Nodes when data grew and decommissioned when the hardware degraded
• Worked with systems engineering team to plan and deploy new Hadoop environments and expand existing Hadoop clusters
• Provided operational support services relating to Hadoop infrastructure and application installation
• Involved in implementing High Availability and automatic failover infrastructure to overcome single point of failure for Name node utilizing zookeeper services
• Worked with cluster users, designers and scientists in troubleshooting map reduce job failures and issues with Hive and YARN
• Supporting Tableau connectivity to Hadoop Cluster using ODBC/JDBC

Confidential - Phoenix, AZ

Hadoop Administrator

Environment: -Hortonworks, Map Reduce, Hive, Pig, Java JDK 1.6, AWS, Cent OS 6.4, Shell Scripting, Flume, Apache, Sqoop, base, Kerberos, Active Directory, Red Hat Linux 6.4, My SQL 5.5

Responsibilities-

• Worked closely with data analysts to construct creative solutions for their analysis tasks.
• Installed, Configured and managed Flume Infrastructure
• Performed data analytics in Hive and then exported this metrics back to Oracle Database using Sqoop
• Created HIVE databases and granted appropriate permissions through Ranger policies.
• Experienced in managing and reviewing Hadoop log files
• Installation of various Hadoop Ecosystems and Hadoop Daemons
• Provided ad-hoc queries and data metrics to the Business Users using Hive, Pig
• Conducting root cause analysis and resolve production problems and data issues
• Proactively involved in ongoing maintenance, support and improvements in Hadoop cluster
• Executed tasks for upgrading cluster on the staging platform before doing it on production cluster
• Monitor cluster stability, use tools to gather statistics and improve performance
• Help to plan for future upgrades and improvements to both processes and infrastructure
• Keep current with latest technologies to help automate tasks and implement tools and processes to manage the environment

Confidential - Dallas, TX

UNIX/LINUX Administrator

Environment: - RedHat Linux, Windows 2008/2010, Tomcat, WebLogic Server administration, Apache, IIS 6.0, Autosys, VERITAS Volume Manager 4.x/5.0

Responsibilities-

• Managed server on VMware provided test environments on virtual machines
• Provided application support to large users groups
• Installed hardware, installed Linux OS, and configured required network on 25 Node HPC cluster
• Configured and Manage Apache web server
• Manage software and hardware RAID systems
• Manage user accounts, user disk usage by setting up quotas
• Manage System Firewall utilizing IP Chains, IP Tables
• Administer System logs, Security logs
• Update software packages and apply security patches
• Write documentation for internal use about System changes, Systems Administration etc.
• Maintained 50 Linux and UNIX servers running high availability application and database for client
• Monitored and maintained all storage devices, created reports, creating vdisks and allocated them to different hosts under the storage network
• Performed storage allocation from EVA storage to match client needs for production and staging environment. Configured LUNs on server end and created file systems as per application/database requirements
• Created CIFS and NFS shares on Linux and HPUX hosts as per client request
• Monitored all environments through Site scope, and other monitoring systems
• Installed application and database packages requested by clients
• Managing RPM and YUM package installations, patch and other servers
• Made changes required by Database and Application team for Linux application servers
• Ran monthly security checks through UNIX and Linux environment and installed security patches required to maintain high security level for our clients
• Monitored and maintained backup and disaster recovery environment created for clients.
• Performed backup and restores for client for production and staging environments
• Copied/moved file systems from one server to another as per client requests
• Configuring the NFS servers, setting up servers in network environment and configuring FTP/NTP/NIS servers, clients for various departments and clients

Confidential

System Administration RedHat-Linux

Environment: -RedHat Linux, Tomcat, WebLogic Server administration, Apache HTTP, VERITAS Volume Manager 4.x/5.0, Oracle 10, MySQL, PHP, Shell Scripting

Responsibilities-

• Worked as a System Administrator on Linux- UNIX platforms
• Involved in maintenance of systems including patching, setting up Print servers, Configuring File systems, using LVM/VERITAS on HPUX/RedHat Linux
• Setting up the Backup solution using native Unix/Linux tools and also worked with Omni back/Data protector solutions
• Installation, configuration and administration of Linux Red Hat, Centos
• Administered Linux servers Linux Red hat - RHEL 6, for several functions including managing Apache/Tomcat server, mail server, MySQL database, and firewalls in both development and production
• Responsibilities as a Unix System Administrator include monitor and tune system to make sure of the optimum level of the performance
• Extensive experience in the concept of LVM, User System Resource Management and Job Scheduling
• Experience in Hardware and Software refreshes on the server
• Performed common system administration task including adding users, creating file systems, configuring volumes weekly mksysb
• Adding Oracle ASM disk to the server. Creating and managing LVM
• Experience in installation and configuration of HBAs and associated firmware/drivers and scanning and configuration of LUNs/Volumes
• Create users with limited and full root privileges. Create and manage sudoers
• Develop Linux shell scripts to accomplish redundant tasks to simplify in distributed administration
• Responsible for resolving network issues using network tools like ping, tcptraceroute, traceroute, tcp dump
• Install and configure various services like DHCP, NFS, DNS, Apache Web Server, NIS, LDAP, Samba, SSH, FTP/SFTP, Sendmail/SMTP, Telnet, RPM Package Management, File System Management
• Perform standard system administration tasks, such as patching, software and hardware upgrades, troubleshooting and problem resolution
• Monitored the servers and Linux scripts regularly and performed troubleshooting steps tested and installed the latest software on server for end-users
• Responsible for Patching Linux Servers
• Perform day to day Linux administration such as user accounts, logon scripts, directory services, file system shares, permissions. Support Oracle database
• Experience in Binding Luns, Naming, and resizing, Unbinding LUNs through Navisphere
• Develop and maintain the documents, library and procedural documents of the system
• Provided 24x7 on call support on a rotation basis