SUMMARY

• Over 8+ years of professional Information Technology experience in Hadoopand Linux Administration activities such as installation, configuration and maintenance of systems/clusters.
• Having 3 years of experience in Linux Administration & Big Data Technologies and 5+ years of experience into Hadoop administration.
• Hands on experience on HadoopClusters using Hortonworks (HDP), Cloudera (CDH4, CDH5), oracle big data and Yarn distributions platforms.Good knowledge on Map R distribution too.
• Possessing skills in Apache Hadoop, Map - Reduce, Pig, Impala, Hive, Platfora, Hbase, Zookeeper, Sqoop, Flume, Kafka, storm, Spark, Datameer, Java Script, and J2EE.
• Experience in deploying and managing the multi-node development and production Hadoopcluster with different Hadoop components (Hive, Pig, Sqoop, Oozie, Flume, HCatalog, Hbase, Zookeeper) using Hortonworks Ambari.
• Good experience in creating various database objects like tables, stored procedures, functions, and triggers using SQL, PL/SQL, and DB2.
• Design and deployment of Storm cluster integration with Kafka and Hbase.
• Experience in Configuring Name-node High availability and Name-node Federation and depth knowledge on Zookeeper for cluster coordination services.
• Experience on Design, configure and manage the backup and disaster recovery for Hadoop data.
• Experience in administering Tableau and Green Plum databases instances in various environments.
• Hands on experience in analyzing Log files for Hadoop and eco system services and finding root cause.
• Experience in importing and exporting the data using Sqoop from HDFS to Relational Database systems/mainframe and vice-versa.
• Worked on NoSQL databases including Hbase, Cassandra and Mongo DB.
• Designing and implementing security for Hadoop cluster with Kerberos secure authentication.
• Hands on experience on Nagios and Ganglia tool for cluster monitoring system.
• Experience in scheduling all Hadoop/Hive/Sqoop/Hbase jobs using Oozie.
• Knowledge of Data Ware Housing concepts and Cognos 8 BI Suit and Business Objects.
• Experience in HDFS data storage and support for running map-reduce jobs.
• Experience in Installing Firmware Upgrades, kernel patches, systems configuration, performance tuning on Unix/Linux systems.
• Expert in Linux Performance monitoring, kernel tuning, Load balancing, health checks and maintaining compliance with specifications.
• Hands on experience in Zookeeper and ZKFC in managing and configuring in NameNode failure scenarios.
• Team Player with good communication and interpersonal skills and also goal oriented approach to problem solving issues.

TECHNICAL SKILLS

Big Data Technologies: Hadoop, HDFS, MapReduce, Yarn, Hive, Pig, Sqoop, Hbase, Flume, Oozie, Spark, Zookeeper.

Hadoop Platforms: Hortonworks and Cloudera

Networking Concepts: OSI Model, TCP/IP, UDP, IPV4, Subnetting, DHCP & DNS

Programming Languages: PIG LATIN, UNIX shell scripting and Bash.

Scripting Languages: PERL and Python

Operating Systems: Linux (CentOS, Ubuntu, Red Hat), Windows, UNIX and Mac OS-X

Database/ETL: Oracle, Cassandra, DB2, MS-SQL Server, MySQL, MS-Access, Hbase, MongoDB, Informatica, Teradata.

XML Languages: XML, DTD, XML Schema, XPath.

Monitoring and Alerting: Nagios, Ganglia, Cloudera Manager, Ambari.

PROFESSIONAL EXPERIENCE:

Confidential

Sr. Hadoop Administrator

Responsibilities:

• Installed, configured and maintained Apache Hadoopclusters for application development of Production, Model - UAT, development and pre-development clusters.
• Managing and scheduling Jobs on HadoopClusters using Cloudera 5.9.0 (CDH5) distributions.
• Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS.
• Developed Hive scripts to combine and find data insights and also create tests cases.
• Developed Yarn dynamic resource pools to run jobs based on users usage across Yarn and Impala.
• Worked on installing Hadoop Ecosystem components such as Sqoop, Pig, Hive, Hbase, Impala, Oozie, Zookeeper and HDFS and knowledge of Mapper/Reduce/HDFS Framework.
• Implemented Fair schedulers on the Job tracker to share the resources of the Cluster for the Map Reduce jobs given by the users.
• Implemented Name Node backup using QJM for High availability.
• Implemented Sentry enterprise security for fine-grained authorization to data and modified ACL’s.
• Implemented security for Hadoopclusters including Open LDAP installation with Active directory, data transport encryption with TLS/SSL.
• Installed and configuring Kerberos KDC installation to implement security to the Hadoop cluster and providing authentication for users.
• Installed OOZIE workflow engine to run multiple Hive and Pig jobs which run independently with time and data availability for analyzing the HDFS audit data.
• Configured Hive and Oozie to store metadata in Microsoft SQL Server.
• Created Hive External tables and loaded the data into tables and query data using HQL.
• Monitored the Hadoop platform through Navigator by analyzing the users accessing the platform and generated audit reports.
• Wrote shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
• Designed and implemented Disaster Recovery Plan for Hadoop Clusters.
• Automated all the jobs for pulling data from FTP server to load data into Hive tables, using Oozie workflows.
• Experience creating real time data streaming solutions using Apache Spark/Spark Streaming Apache Storm, Kafka and Flume.
• Monitored the Hadoop platform through Navigator by analyzing the users accessing the platform and generated audit reports.
• Worked on handling Hive queries using Spark SQL that integrates with Spark environment implemented in Scala.
• Handle the data exchange between HDFS & Web Applications and databases using Flume and Sqoop.
• Work closely with Hadoop developers, Architects, Business users, designers in troubleshooting job failures and issues and helping to developers.
• Working with data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos principals, generating key tabs and testing MFS, and Hive.
• Worked on setting up High availability for major production cluster and designed automatic fail over control using zookeeper and quorum journal nodes.
• Worked on importing and exporting data from Oracle and DB2 into HDFS using Sqoop and loading data into HDFS.
• Worked with Infrastructure teams to install operating system, Hadoop updates, patches, version upgrades as required.
• Worked with different file formats such as Text, Sequence files, Avro, ORC and Parquet.
• Good understanding of Scrum methodologies, Test Driven Development and continuous integration.
• Involved in various POC activity using technology like Map reduce, Hive, Pig, and Oozie.
• Involved in log file management where the logs greater than 7 days old were removed from log folder and loaded into HDFS and stored for 3 months.

Environment: Cloudera, CDH 5.9.0, Cloudera manager, Hortonworks Ambari, Sqoop, Flume, Hive, Spark, Scala, Hbase, HQL, Pig, RHEL 6.5, Oracle, MS-SQL, Zookeeper, Oozie, MapReduce, Apache Hadoop 2.x, Python, Postgresql, Ganglia and Nagios.

Confidential, Indianapolis, IN

Sr. Hadoop Administrator

Responsibilities:

• As an Hadoop admin worked in Huge Cluster on maintaining nodes with High availability environment using Hortonworks Ambari manager and Cloudera Manager.
• Involved in Installation and configuration, Hadoop Cluster and Maintenance, Cluster Monitoring, Troubleshooting and Transform data from RDBMS to HDFS and followed proper backup & Recovery strategies.
• Analyzed the data by performing Hive queries and running Pig scripts to know user behavior like frequency of calls, top calling customers and designed and implemented service layer over Hbase Database.
• Handled importing of data from various data sources, performed transformations using Hive, MapReduce and loaded data into HDFS.
• Provide Business Intelligence support using Tableau for implementing effective Business dashboards & visualizations of data.
• Configuring, implementing and supporting High Availability (Replication) with Load balancing (Sharding) cluster of MongoDB having Terabytes of data.
• Hadoop cluster monitoring and troubleshooting Hive, Datameer, Platfora and flume.
• Experience with securing Hadoopclusters including Kerberos KDC installation, OpenLDAP installation, data transport encryption with TLS.
• Implemented a distributed messaging queue to integrate with Cassandra using Apache Kafka and Zookeeper.
• Used Cassandra in multiple virtual and physical data centers to ensure the system was highly redundant and scalable.
• Exported the analyzed data from MySQL to the HDFS using Sqoop for visualization and to generate reports for the BI team.
• Importing of data from various data sources such as Oracle and Comptel server into HDFS using transformations such as Sqoop, Map Reduce.
• Developed various dashboards in Tableau, used context filters, sets while dealing with huge volume of data.
• Designed and developed scalable and custom Hadoop solutions as per dynamic data needs and coordinated with technical team for production deployment of software applications for maintenance.
• Data is loaded back to the Teradata for the BASEL reporting and for the business users to analyze and visualize the data using Datameer.
• Real time streaming data using Spark with Kafka.
• Implemented a different use case to run applications in YARN containers as long running jobs.
• Worked with ETL team to load data into Data Warehouse/Data Marts using Informatica.
• Experience in providing support to data analyst in running Pig and Hive queries.
• Experience in administering the Linux systems to deploy Hadoop cluster and monitoring the cluster using Nagios and Ganglia. Reviewing the log files and error solving.
• Involved in importing the real time data to Hadoop using Kafka.
• Data Standardization using Talend components like tmap, tloqateaddress, tmatchgrouping, trecordmatch, tnormalize, tdenormalize and loading the data into MDM Server by creating Data models, SQL Server Database Table and Archive Tables.
• Developed custom Process chains to support master data and transaction data loads from BI to BPC.
• Involved in various POC activity using technology like Map reduce, Hive, Pig, and Oozie.

Environment:Hadoop, HDFS, Hive, Sqoop, Flume, Hortonworks, Cassandra, Java, Impala, Talend, Tableau, Kafka, storm, Zookeeper and Hbase, Kafka, YARN, Oracle 9i/10g/11 RAC with Solaris/RedHat, MongoDB, Kerberos, SQL plus, PHP, Shell Scripting, ETL/BI architectures and SQL, RedHat/Suse Linux, EM Cloud Control.

Confidential - Houston, TX

Hadoop Administrator

Responsibilities:

• Installed/Configured/Maintained Apache Hadoop and Cloudera Hadoop clusters for application development and Hadoop tools like Hive, Pig, Hbase, Zookeeper and Sqoop.
• Wrote the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
• Managing and scheduling Jobs on a Hadoop cluster.
• Worked on Providing User support and application support through remedy ticket management system on Hadoop Infrastructure.
• Installed and configured Hadoop cluster in Development, Testing and Production environments.
• Performed both major and minor upgrades to the existing CDH cluster.
• Responsible for monitoring and supporting Development activities.
• Installation of various Hadoop Ecosystems and Hadoop Daemons.
• Installed and configured flume agents with well-defined sources, channels and sinks.
• Configured safety valve to create active directory filters to sync the LDAP directory for Hue.
• Developed scripts to delete the empty Hive tables existing in the Hadoop file system.
• Understanding the existing Enterprise data warehouse set up and provided design and architecture suggestion converting to Hadoop using MapReduce, HIVE, SQOOP and Pig Latin.
• Implemented Name Node backup using NFS. This was done for High availability.
• Worked on importing and exporting data from Oracle and DB2 into HDFS and HIVE using Sqoop.
• Wrote shell scripts for rolling day-to-day processes and it automated using Crontab.
• Collected the logs data from web servers and integrated in to HDFS using Flume.
• Implemented FIFO schedulers on the Job tracker to share the resources of the Cluster for the MapReduce jobs given by the users.
• Involved in Data model sessions to develop models for HIVE tables.
• Involved in gathering business requirements and analysis of business use cases.

Environment: Apache Hadoop, CDH4, Hive, Hue, Pig, Hbase, MapReduce, Sqoop, RedHat, CentOS and Flume.

Confidential - Reston, VA

Linux/Hadoop Administrator

Responsibilities:

• Responsible for installation and configuration of RedHat 4.x, 5.x and SUSE Linux Enterprise Server (SLES) 10.x, 11.0 on IBM and HP hardware.
• Installed IBM Systems director client on various servers and maintained the IBM Systems director for Hardware monitoring.
• Manage Virtual servers on VMwareESXServer3.0/3.5.
• Experienced in deploying, cloning and creating template of virtual machines using VMware.
• Performed all duties related to system administration like troubleshooting, providing sudo access, modifying DNS entries, NFS backup recovery (scripts).
• Implemented Relax and Recover (ReaR) for disaster recovery.
• Updated firmware on Servers, Installed patches and packages for security vulnerabilities for Linux.
• Monitored system resources, like network, logs, disk usage etc.
• Learned and managed systems with the Puppet configuration management tool.
• Built 3 and 4 node Oracle RAC ServersConfigured SUDO to provide SUDO access to various users and group for elevated access.
• Worked on tools like Yum, YaST, RPM, Zypper, rug for Package management.
• Deployed latest patches and performed RedHat Linux Kernel Tuning for Oracle.
• Installed LAMP server (RHEL 5.4, Apache 2.2, MySQL 5.0 server and Perl).
• Used the LVM extensively for Creating/Modifying/Deleting Physical Volume, Volume Groups, and Logical Volumes and file systems.
• Experience with IBM Power7 Virtualization and XEN Virtualization, VMware.
• Creating Cron jobs and giving Cron eligibility for administrative accounts.
• Performance Monitoring and Tuning using iostat, vmstat, netstat, sar, ksar, top.
• Provided 24x7 on call support for production environment.
• Resolving issues as they arise with JIRA and Confluence in coordination with the IT department.
• Completed a full migration of UNIX Server (AIX and Linux), including all network services.

Environment: RedHat Linux, Suse Linux Enterprise server, UNIX, puppet, VMware, Oracle, Tivoli, Kernel Tuning, LVM

Confidential

Linux Administrator

Responsibilities:

• Installing, configuring and upgrading Linux (Primarily REDHAT and UBUNTU) and Windows Servers.
• Co-coordinating with the customers vendors for any system up gradation and giving the exact procedure to follow up.
• Contact various systems administration works under CentOS, Red Hat Linux environments.
• Configured Domain Name System (DNS) for hostname to IP resolution.
• Creating users, assigning groups and home directories, setting quota and permissions; administering file systems and recognizing file access problems.
• Installed Cent OS using Pre-Execution environment boot and Kick start method on multiple servers, remote installation of Linux using PXE boot.
• Performed various configurations which include networking and IPTable, resolving host names and SSH keyless login.
• Maintained and monitored all servers' operating system, managing Disk File systems, server performance, application patch level, disk space and memory usage, user activities on daily basis, also RAID configurations.
• Installed and verified that all AIX/Linux patches are applied to the servers.
• Responsible for maintenance Raid-Groups, LUN Assignments as per agreed design documents.
• Maintenance and installation of RPM and YUM package installations and other server management.
• Performed various configurations which include networking and IPTable, resolving hostnames, SSH key less login.
• Developed and optimize physical design of MySQL database systems.
• Implemented new releases to add more functionality as per the requirements.
• Supported LANs, WANs, network segments, Internet, and intranet systems and maintaining the integrity of the network.
• Schedule backup jobs by implementing Cron job schedule during non-business hour.
• Managing Disk File Systems, Server Performance, Users Creation and Granting file access Permissions and RAID configurations.
• Troubleshooting and fixing the issues at User level, System level and Network level by using various tools and utilities.

Environment: YUM, RAID, MYSQL 5.1.4, PHP, SHELL SCRIPT, MYSQL, WORKBENCH, LINUX 5.0, 5.1, CentOS, AIX

Confidential

Linux Administrator

Responsibilities:

• Installed, configured, maintained and administrated the network servers DNS, NFS and application servers Apache and Samba server.
• Installation and Configuration of SSH, TELNET, FTP, DHCP, DNS.
• Worked on UNIX shell scripting for system/application in automating server task, installing and monitoring applications and data feeding file transfer and log files.
• Maintained UNIX (Red Hat Enterprise Linux4, 5, CentOS4, 5, VMware) on SunEnterprise servers & Dell Servers.
• Implemented the Jumpstart servers and Kickstart Servers to automate the server builds for multiple profiles.
• Installed and deployed RPM Packages.
• APACHE Server Administration with Virtual Hosting.
• Worked extensively in using VI editor to edit necessary files writing shell script.
• Worked on adding new Users and +groups and give sudo access and central file synchronization via sudoers, authorized keys, password, shadow, and group.
• Coordinated with application team in installation, configuration and troubleshoot issues with Apache, Web logic on Linux servers.
• Local and Remote administering of servers, routers and networks using Telnet and SSH.
• Monitored client disk quotas &disk space usage.
• Worked on backup technologies like VeritasNetbackup4.x, 5.0, 6.x and Tivoli Storage Manager 5.5.
• Involved in back up, firewall rules, LVM configuration, monitoring servers and on call support.
• Created BASH shell scripts to automate Cron jobs and system maintenance. Scheduled Cron jobs for job automation.

Environment: RedHat 4/5, Solaris 8/9/10, CentOS 4/5, SUSE Linux 10.1/10.3, VMware