SUMMARY

• Around 7 years of System Administrator experience in IT background which includes around 3+ years of experience with emphasis on Hadoop Technologies.
• Strong Knowledge on Multi Clustered environment and setting up Cloudera Hadoop Eco - System. Experience in installation, configuration and management of Hadoop Clusters.
• Experience writing Map Reduce Jobs, HIVEQL
• Experience with Hbase, PIG, NoSQL and MongoDB.
• Set up and configure Hadoop System on Amazon AWS for processing massive volumes of Data
• Work with Enterprise Analytics team and transform analytics requirements into Hadoop centric technologies.
• Monitor Map Reduces and analyze cluster performance.
• Performance tuning the Hadoop Cluster by gathering and analyzing the existing infrastructure
• Automating the Hadoop Installation, configuration and maintaining the Cluster by using the tools like puppet.
• Setting up monitoring infrastructure for Hadoop Cluster using Nagios and Ganglia.
• Working with Flume to load the log Data from multiple sources directly into HDFS
• Configuring the Zookeeper to coordinate the servers in Clusters and to maintain the Data consistency.
• Data migration from existing Data stores to Hadoop using Sqoop.
• Upgrade and maintenance of Apache product stack.
• Experience in managing Hadoop infrastructure like commissioning, decommissioning, log rotation, rack topology implementation.
• Designing both time driven and Data driven automated workflows using Oozie.
• Supporting analysts by Administering and Configuring HIVE.
• Providing support to Data analyst in running PIG and HIVE queries.
• Writing shell scripts to dump the Shared Data from MySQL servers to HDFS.
• Familiar with Java virtual machine (JVM) and multi-threaded processing.
• Linux Certified with hands-on experience in installing, patching, upgrading and configuring Linux based operating systems - RHEL 6, Ubuntu, CentOS in a large set of clusters.

TECHNICAL SKILLS

Big Data Ecosystem: HDFS, HBase, Hadoop MapReduce, Zookeeper, Hive, Pig, Sqoop,Flume, Oozie, Spark

Paradigms: MapReduce & YARN

Hadoop Distributions: Cloudera, Hortonworks

RDBMS/Database: ORACLE 10g/9i, MySQL

Scripting Languages: Shell scripting, UNIX shell scripting

Operating Systems: UNIX, Linux, Windows

Methodologies: SDLC Practice Areas, Agile Software Development

Monitoring Tools: Ganglia, Nagios, Ambari

DevOps Tools: Chef, AWS, Kafka

PROFESSIONAL EXPERIENCE

Confidential, Westlake, TX

Hadoop Administrator

Responsibilities:

• Installed and configured various components of Hadoop ecosystem and maintained their integrity
• Planning for production cluster hardware and software installation on production cluster and communicating with multiple teams to get it done
• Designed, configured and managed the backup and disaster recovery for HDFS data
• Commissioned Data Nodes when data grew and decommissioned when the hardware degraded
• Migrated data across clusters using DISTCP
• Experience in collecting metrics for Hadoop clusters using Ganglia and Ambari
• Worked with systems engineering team to plan and deploy new Hadoop environments and expand existing Hadoop clusters
• Monitored multiple Hadoop clusters environments using Ganglia and Nagios. Monitored workload, job performance and capacity planning using Ambari.
• Focused on configuration management and automation using CHEF.
• Used Apache Kafka in handling real time data feeds.
• Worked with application teams to install Hadoop updates, patches, version upgrades as required.
• Installed and configured Hive, Pig, Sqoop and Oozie on the HDP cluster
• Involved in implementing High Availability and automatic failover infrastructure to overcome single point of failure for Name node utilizing zookeeper services
• Implemented HDFS snapshot feature
• Performed a Major upgrade in production environment from HDP 1.3 to HDP 2.0
• Worked with big data developers, designers and scientists in troubleshooting map reduce job failures and issues with Hive, Pig and Flume
• Configured custom interceptors in Flume agents for replicating and multiplexing data into multiple sinks
• Administrating Tableau Server backing up the reports and providing privileges to users
• Worked on Tableau for generating reports on HDFS data
• Installed Ambari on existing Hadoop cluster

Environment: Hadoop, HDFS, Ambari, Map reduce, Yarn, Oracle 11g/10g, Big Data Cloudera CDH Apache Hadoop, Apache Kafka, Chef, SQL plus, Shell Scripting, Red Hat Linux, EM Cloud Control.

Confidential, Boston MA

Hadoop Administrator

Responsibilities:

• Worked on performing major upgrade of cluster from CDH3u6 to CDH4.2.0.
• Implemented Namenode High Availability on the Hadoop cluster to overcome single point of failure.
• Installed Cloudera Manager on an already existing Hadoop cluster.
• Involved in efficiently collecting and aggregating large amounts of streaming log data into Hadoop Cluster using Apache Flume.
• User behavior and their patterns were studied by performing analysis on the data stored in HDFS using Hive.
• Used HiveQL to write Hive queries from the existing SQL queries.
• Responsible for designing and implementing ETL process to load data from different sources, perform data mining and analyze data using visualization/reporting tools to leverage the performance of OpenStack
• The analyzed data mined from huge volumes of data was exported to MySQL using Sqoop.
• Developed custom MapReduce programs and custom User Defined Functions (UDFs) in Hive to transform the large volumes of data with respect to business requirement.
• Involved in installing and configuring Kerberos to implement security to the Hadoop cluster and providing authentication for users.
• Used CHEF as part of Continuous Integration/ Continuous Deployment (CI/CD).
• Worked on installation of DataStax Cassandra cluster.
• Worked with Big Data Analysts, Designers and Scientists in troubleshooting map reduce job failures and issues with Hive, Pig, and Flume etc.

Environment: Hadoop, MapReduce, Hive, Oozie, Sqoop, Flume, Cloudera Manager, Chef, Shell Script

Confidential, San Bruno, CA

Linux System Administrator

Responsibilities:

• Performed installation, configuration and maintenance of Redhat Linux 4.x/ 5.x/6.x
• Installation, configuration and Operating System upgrade on Sun Solaris 8,9,10, on sun Servers
• Also excelled on Linux kernel, memory upgrades, and swap area. Redhat Linux Kickstart Installation. Configuring DNS, DHCP, NIS, NFS in Redhat Linux other Network Services.
• Performed daily backups, restoration of data as per requests from developers using Veritas Netbackup.
• Veritas File system and Veritas Volume Manager 2.5 are used to configure the RAID 1 and RAID 5 Storage Systems
• Experience in Servers consolidation and virtualization using VMware virtual infrastructure, VMware ESX.
• Wrote Shell Scripts as cron jobs and deployment scripts for various reasons.
• Mounted Disks allocated from the SAN (Clarion) for Linux and Solaris servers.
• Installed new and removed bad HBA cards from the server. Moved the Volumes from CX600 to CX700 to increase storage capacity.
• Provided application support for Weblogic application srvers as required.
• Network trouble shooting using ‘ndd’, 'traceroute', 'netstat', 'ifconfig' and 'snoop' etc
• Installed of all the required Software, patches, documentation, licensing trouble shooting.
• Troubleshoot network & DNS issues and handled VERITAS Netbackup server & client connectivity problems.
• Monitored overall system performance, performed user management, system updates and disk & storage management.
• Created users, manage user permissions, maintain User & File System quota on Redhat Linux and Solaris.
• Production support of Apache and JBoss including installation, configuration, management and troubleshooting
• Setup of full networking services and protocols on UNIX including NIS/NFS, DNS, SSH, DHCP, NIDS, TCP/IP, ARP, applications, kickstart and print servers to insure optimal networking, application, and printing functionality.
• Installed and configured Sudo for users to access the root privileges.
• Used Zone Configuration in solaris10 boxes. Used zoneadm utility to create and maintain the zones. Created many number of zones under the Global zone and setup network for the zones.
• Wrote Technical documentation for various projects and worked with Technical writer to finalize the documents.

Environment: Solaris 8,9,10, Redhat Linux 4.x,5.x,6.x VMware ESX Server 3.5.0, JBOSS, Remedy, EMC Power Path, windows 2003/2007/XP/VISTA/7, Solstice Disk Suite, AIX 6.1, 5.3, Veritas Cluster Server, Enterprise Security Manager, Opsware, Weblogic 8.1, Oracle 10g/11g.

Confidential, Westborough, MA

System Administrator

Responsibilities:

• Installation and configuration of Red Hat Linux, Solaris, Fedora and CentOS on new server builds as well as during the upgrade situations.
• Log management like monitoring and cleaning the old log files.
• System audit report like no. of logins, success & failures, running cron jobs.
• System performance for hourly basis or daily basis.
• Remotely coping files using sftp, ftp, scp, winscp, and filezilla.
• Created user roles and groups for securing the resources using local operating System authentication.
• Experienced in tasks like managing User Accounts and Groups, managing Disks and Filesystems.
• Install and configure Instruction Detection System (IDS) like Tripwire, Snort, and Lids.
• Configuring & monitoring DHCP server.
• Taking backup using tar and recovering during the data loss.
• Experience in writing bash scripts for job automation.
• Documenting the installation of third-party software’s.
• Configuring printers to the Solaris and Linux servers and also installing third party softwares.
• Maintaining relations with project managers, DBA’s, Developers, Application support teams and operational support teams to facilitate effective project deployment.
• Manage system installation, troubleshooting, maintenance, performance tuning, managing storage resources, network configuration to fit application and database requirements.
• Responsible for modifying and optimizing backup schedules and developing shell scripts for it.
• Performed regular installation of patches using RPM and YUM.
• Maintained LVM, VxVM and SVM filesystems along with NFS.

Environment: Linux ( Red Hat Enterprise, CentOS), Windows 2000/NT, HP, IBM, Solaris, Oracle 8i, Cisco routers/switches, Dell 6400, 1250, Sun E450, E250.