SUMMARY:

• Around 5 years of professional experience in IT industry in which includes 2 +years of experience in Big Data Hadoop Ecosystems and 2 years as Linux Admin.
• Experience in designing, installation, configuration and management of Cloudera and Horton works apache Hadoop Distribution.
• Experience in deploying and managing the multi - node development, testing and production Hadoop cluster with different Hadoop components (HIVE, PIG, SQOOP, OOZIE, FLUME, HBASE, ZOOKEEPER) using Cloudera Manager and Apache Ambari.
• Installing and configuring Kafka cluster and monitoring the cluster using Nagios and Ganglia.
• Experience in configuring Zookeeper to provide High Availability and Cluster service co-ordination.
• Configuring Kerberos and integrating with Directory services.
• Providing security for Hadoop Cluster with Kerberos, Active Directory/LDAP, and TLS/SSL utilizations and dynamic tuning to make cluster available and efficient.
• Expertise in Managing, Monitoring and Administration of Hadoop for Multi Hundred Node Cluster with different distributions like Cloudera CDH and Horton works HDP.
• Experience in managing the cluster resources by implementing fair scheduler and capacity scheduler.
• Worked with Sqoop for importing and exporting data from different databases like RDBMS, Oracle, and MySQL into HDFS.
• Worked with Flume for collecting the logs from log collector into HDFS.
• Experience in developing and scheduling ETL workflows in Hadoop using Oozie.
• Experienced with different kind of compression techniques like LZO, GZIP, and Snappy.
• Involved in customer interactions, business user meetings, vendor calls and technical team discussions to take right choices in terms of design and implementations and to provide best practices for the organization.
• Worked on Disaster Management with Hadoop cluster.
• Experience in performing minor and major upgrades, commissioning and decommissioning of data nodes on Hadoop cluster.
• Experience with NoSQL databases like HBase and Cassandra.
• Installation, patching, upgrading, tuning, configuring and troubleshooting Linux based operating systems Red Hat and Centos and virtualization in a large set of servers.
• Experience in Install and configuration of Web hosting administration.
• Experience in Installation of VMware ESX server and creation of VMs and install different guest OS.
• Experience with system integration, capacity planning, performance tuning, system monitoring, system security, operating system hardening and load balancing.
• Experience in Managing and Scheduling Cron jobs such as enabling system logging, network logging of servers for maintenance, performance tuning and testing.
• Experience in Managing various Network related tasks such as TCP/IP, NFS, DNS, DHCP and SMTP.
• Proficient in OS upgrades and Patch loading as and when required.
• Expert in setting up SSH, SCP and VSFTP connectivity between UNIX hosts.
• Experience in supporting users to debug their job failures.
• Experience in supporting systems with 24X7 availability and monitoring

PROFESSIONAL EXPERIENCE:

Confidential, Austin, TX

Linux and Hadoop Admin

Responsibilities:

• Configuring, Maintaining and Monitoring Hadoop Cluster using Cloudera Manager.
• Monitoring Hadoop Productions Clusters using Cloudera Manager and 24x7 on call support.
• Performed both major and minor upgrades to the existing Cloudera Hadoop cluster.
• Upgraded Cloudera manger from 5.8 to 5.12.
• Applied patches and bug fixes on Hadoop Clusters.
• Day to day responsibilities includes solving developer issues, deployments moving code from one environment to other environment, providing access to new users and providing instant solutions to reduce the impact and documenting the same and preventing future issues.
• Installed non-Hadoop services on the production servers.
• Troubleshooting HBase issues.
• Kernel Patching on data nodes using BMC tools.
• Request vendors (HP&Dell) to replace failures hardware on servers.
• File system creation and extension.
• Commissioning and decommission of nodes on Hadoop.
• Involved in all maintenance activities of Hadoop Productions Clusters.
• Debugging issues and staring for non-Hadoop services.
• Troubleshooting Cluster issues and preparing run books.
• Reviewing and on boarding applications to Cluster.
• Worked on Providing User support and application support on Hadoop Infrastructure.
• Implemented schedulers on the Resource Manager to share the resources of the cluster.

Environment: HDFS, Map reduce Yarn, HBase, Hive, Kafka, Spark, Kerberos, Pig, Sqoop, Solr, Cloudera mangers services using Cloudera Manager.

Confidential, Houston, TX

Hadoop Engineer

Responsibilities:

• Configuring, Maintaining, and Monitoring Hadoop Cluster using Apache Ambari, Hortonworks distribution of Hadoop.
• Installed HDP 2.5on 100 node cloud cluster using Apache Ambari 2.4 and Set Up SSL for Ambari.
• Enable High Availability Name Node, Resourcemanager, HBase and HiveServer2 automatic failover infrastructure to overcome single point of failure.
• Installed and configure various Services of Hadoop Ecosystems using Apache Ambari.
• Ambari Up gradation from 2.3 to 2.4.and HDP Up gradation from HDP2.4 to 2.5.
• Installed Kerberos using Ambari and created and update the Kerberos keytab and Principle for Service accounts.
• Installing Ranger and Enable the HDFS and Hive Plugins. Enable Ranger Plugins on a Kerberos cluster.
• Creating and updating ranger policies using Apache Ranger.
• Installed Mysql server and configure the Slave and Master replication.
• Develop the Script for Mysql database backup. Schedule backup using Falcon.
• Commissioning and Decommissioning Hadoop Cluster nodes Including Balancing HDFS block data.
• Part of planning/migration team for Application Migration from MapR distribution to HDP environment.
• Developed PIG scripts to cleanse the data for removing new line characters and null values removal.
• Crated HIVE Internal and External tables using Partitioning for storing cleansed data and implemented incremental logic to store the delta data onto Hive tables.
• Reviewing application architectures for better understanding of the dependencies, file formats, types of data, tools, service-accounts etc.., i.e. important factors in order to migrate the apps to HDP platform.
• Ingested various types of data into Hive using ELake Ingestion Framework which internally uses Pig, Hive and Spark for data processing.
• Worked with Avro schemas for Hive. Created Hive tables on top of HBase using Storage Handler for effective OLAP analysis.
• Import and export hive tables and Hbase Snapshot.
• Used SQOOP for ingesting Structured data onto HDFS system
• Creating queues on YARN queue manager to share the resources of the Cluster for the Map Reduce jobs given by the users.
• Performed cluster back using DISTCP.
• Developed script to check the 777 Permission of the Hive and HBase tables.
• Developed script to check the Disk Space utilization of user home directories on Gateway servers and Master Nodes
• Monitoring Hadoop jobs using oozie and falcon.
• HDFS and HIVE mirroring using Apache Falcon.
• Inputs to development regarding the efficient utilization of resources like memory and CPU utilization based on the running statistics of Map and Reduce tasks.
• Changes to the configuration properties of the cluster based on volume of the data being processed and performance of the cluster.
• Discussions with other technical teams on regular basis regarding upgrades, Process changes, any special processing and feedback.

Environment: HDFS, Map Reduce Yarn, Hive, PIG, Sqoop, Oozie, Falcon, HBase, Zookeeper, Zeppelin, Ambari infra, Kafka, Kerberos and Ranger using Ambari.

Confidential

Hadoop Admin

Responsibilities:

• Configuring, Maintaining, and Monitoring Hadoop Cluster using Cloudera manager.
• Installed, configured and deployed a 20 node Cloudera Hadoop cluster for development.
• Commissioning and decommissioning of nodes on Hadoop cluster.
• Performed both major and minor upgrades to the existing CDH cluster.
• Performance tune Hadoop cluster to achieve higher performance
• Configured Hive metastore with MYSQL, which stores the metadata of Hive tables
• Benchmarking Hadoop clusters using DFSIO, Teragen, and Terasort.
• Used Ganglia and Nagios for monitoring the cluster around the clock
• Moved data from HDFS to RDBMS and vice-versa using SQOOP.
• Used Hive and created Hive tables and involved in data loading.
• Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
• Set up automated processes to analyze the System and Hadoop log files for predefined errors and send alerts to appropriate groups.

Environment: HDFS, Map Reduce, Hive, Oozie, Pig, Flume, Sqoop, Kerberos, Zookeeper and Mysql using Cloudera Manager.

Confidential

System Admin

Responsibilities:

• Administration of RHEL, which includes installation, testing, tuning, upgrading and loading patches, troubleshooting both physical and virtual server issues.
• Creating, cloning Linux Virtual Machines.
• Installing Red Hat Linux using kick start and applying security polices for hardening the server based on the company policies.
• RPM and YUM package installations, patch and other server management.
• Managing systems routine backup, scheduling jobs like disabling and enabling cron jobs, enabling system logging, network logging of servers for maintenance, performance tuning, testing.
• Tech and non-tech refresh of Linux servers, which includes new hardware, OS, upgrade, application installation, testing.
• Set up user and group login ID's, printing parameters, network configuration, password, resolving permissions issues, and user and group quota.
• Installing MySqlDB in Linux and Customize the MySQL DB parameters.
• Working with Service Now incident tool.
• Creating physical volumes, volume groups and logical volumes.
• Samba Server configuration with Samba Clients.
• Knowledge of IP tables, SELINUX.
• Modified existing Linux file systems to a Standard EXT3.
• Configuration and administration of NFS, FTP, SAMBA, NIS.
• Maintenance of DNS, DHCP and APACHE services on Linux machines.
• Gathering requirements from customers and business partners and design, implement and provide solutions in building the environment.
• Installing and configuring Apache and supporting them on Linux production servers.

Environment: Red-Hat Linux Enterprise servers (HP Proliant DL 585, BL ... ML Series, SAN (Netapp), VERITAS Cluster Server 5.0, Windows 2003 server, Shell programming.