SUMMARY:

• Around 5 years of professional experience in IT industry in which includes 2 +years of experience in Big Data Hadoop Ecosystems and 2 years as Linux Admin.
• Experience in designing, installation, configuration and management of Cloudera and Horton works apache Hadoop Distribution.
• Experience in deploying and managing teh multi - node development, testing and production Hadoop cluster wif different Hadoop components (HIVE, PIG, SQOOP, OOZIE, FLUME, HBASE, ZOOKEEPER) using Cloudera Manager and Apache Ambari.
• Installing and configuring Kafka cluster and monitoring teh cluster using Nagios and Ganglia.
• Experience in configuring Zookeeper to provide High Availability and Cluster service co-ordination.
• Configuring Kerberos and integrating wif Directory services.
• Providing security for Hadoop Cluster wif Kerberos, Active Directory/LDAP, and TLS/SSL utilizations and dynamic tuning to make cluster available and efficient.
• Expertise in Managing, Monitoring and Administration of Hadoop for Multi Hundred Node Cluster wif different distributions like Cloudera CDH and Horton works HDP.
• Experience in managing teh cluster resources by implementing fair scheduler and capacity scheduler.
• Worked wif Sqoop for importing and exporting data from different databases like RDBMS, Oracle, and MySQL into HDFS.
• Worked wif Flume for collecting teh logs from log collector into HDFS.
• Experience in developing and scheduling ETL workflows in Hadoop using Oozie.
• Experienced wif different kind of compression techniques like LZO, GZIP, and Snappy.
• Involved in customer interactions, business user meetings, vendor calls and technical team discussions to take right choices in terms of design and implementations and to provide best practices for teh organization.
• Worked on Disaster Management wif Hadoop cluster.
• Experience in performing minor and major upgrades, commissioning and decommissioning of data nodes on Hadoop cluster.
• Experience wif NoSQL databases like HBase and Cassandra.
• Installation, patching, upgrading, tuning, configuring and troubleshooting Linux based operating systems Red Hat and Centos and virtualization in a large set of servers.
• Experience in Install and configuration of Web hosting administration.
• Experience in Installation of VMware ESX server and creation of VMs and install different guest OS.
• Experience wif system integration, capacity planning, performance tuning, system monitoring, system security, operating system hardening and load balancing.
• Experience in Managing and Scheduling Cron jobs such as enabling system logging, network logging of servers for maintenance, performance tuning and testing.
• Experience in Managing various Network related tasks such as TCP/IP, NFS, DNS, DHCP and SMTP.
• Proficient in OS upgrades and Patch loading as and when required.
• Expert in setting up SSH, SCP and VSFTP connectivity between UNIX hosts.
• Experience in supporting users to debug their job failures.
• Experience in supporting systems wif 24X7 availability and monitoring

TECHNICAL SKILLS:

Hadoop Ecosystem Components: HDFS, Map reduce YARN, Hive, Pig, Sqoop, Oozie, Flume and Zookeeper.

No SQL Databases: HBase and Cassandra.

Monitoring Tools: Ganglia and Nagios.

Security: Kerberos, sentry and Ranger.

Scripting languages: Bash and Python.

RDBMS Databases: Oracle Sql, PL/SQL, MySql and SqlServer.

PROFESSIONAL EXPERIENCE:

Confidential, Austin, TX

Linux and Hadoop Admin

Responsibilities:

• Configuring, Maintaining and Monitoring Hadoop Cluster using Cloudera Manager.
• Monitoring Hadoop Productions Clusters using Cloudera Manager and 24x7 on call support.
• Performed both major and minor upgrades to teh existing Cloudera Hadoop cluster.
• Upgraded Cloudera manger from 5.8 to 5.12.
• Applied patches and bug fixes on Hadoop Clusters.
• Day to day responsibilities includes solving developer issues, deployments moving code from one environment to other environment, providing access to new users and providing instant solutions to reduce teh impact and documenting teh same and preventing future issues.
• Installed non-Hadoop services on teh production servers.
• Troubleshooting HBase issues.
• Kernel Patching on data nodes using BMC tools.
• Request vendors (HP&Dell) to replace failures hardware on servers.
• File system creation and extension.
• Commissioning and decommission of nodes on Hadoop.
• Involved in all maintenance activities of Hadoop Productions Clusters.
• Debugging issues and staring for non-Hadoop services.
• Troubleshooting Cluster issues and preparing run books.
• Reviewing and on boarding applications to Cluster.
• Worked on Providing User support and application support on Hadoop Infrastructure.
• Implemented schedulers on teh Resource Manager to share teh resources of teh cluster.

Environment: HDFS, Map reduce Yarn, HBase, Hive, Kafka, Spark, Kerberos, Pig, Sqoop, Solr, Cloudera mangers services using Cloudera Manager.

Confidential, Houston, TX

Hadoop Engineer

Responsibilities:

• Configuring, Maintaining, and Monitoring Hadoop Cluster using Apache Ambari, Hortonworks distribution of Hadoop.
• Installed HDP 2.5on 100 node cloud cluster using Apache Ambari 2.4 and Set Up SSL for Ambari.
• Enable High Availability Name Node, Resourcemanager, HBase and HiveServer2 automatic failover infrastructure to overcome single point of failure.
• Installed and configure various Services of Hadoop Ecosystems using Apache Ambari.
• Ambari Up gradation from 2.3 to 2.4.and HDP Up gradation from HDP2.4 to 2.5.
• Installed Kerberos using Ambari and created and update teh Kerberos keytab and Principle for Service accounts.
• Installing Ranger and Enable teh HDFS and Hive Plugins. Enable Ranger Plugins on a Kerberos cluster.
• Creating and updating ranger policies using Apache Ranger.
• Installed Mysql server and configure teh Slave and Master replication.
• Develop teh Script for Mysql database backup. Schedule backup using Falcon.
• Commissioning and Decommissioning Hadoop Cluster nodes Including Balancing HDFS block data.
• Part of planning/migration team for Application Migration from MapR distribution to HDP environment.
• Developed PIG scripts to cleanse teh data for removing new line characters and null values removal.
• Crated HIVE Internal and External tables using Partitioning for storing cleansed data and implemented incremental logic to store teh delta data onto Hive tables.
• Reviewing application architectures for better understanding of teh dependencies, file formats, types of data, tools, service-accounts etc.., me.e. important factors in order to migrate teh apps to HDP platform.
• Ingested various types of data into Hive using ELake Ingestion Framework which internally uses Pig, Hive and Spark for data processing.
• Worked wif Avro schemas for Hive. Created Hive tables on top of HBase using Storage Handler for TEMPeffective OLAP analysis.
• Import and export hive tables and Hbase Snapshot.
• Used SQOOP for ingesting Structured data onto HDFS system
• Creating queues on YARN queue manager to share teh resources of teh Cluster for teh Map Reduce jobs given by teh users.
• Performed cluster back using DISTCP.
• Developed script to check teh 777 Permission of teh Hive and HBase tables.
• Developed script to check teh Disk Space utilization of user home directories on Gateway servers and Master Nodes
• Monitoring Hadoop jobs using oozie and falcon.
• HDFS and HIVE mirroring using Apache Falcon.
• Inputs to development regarding teh efficient utilization of resources like memory and CPU utilization based on teh running statistics of Map and Reduce tasks.
• Changes to teh configuration properties of teh cluster based on volume of teh data being processed and performance of teh cluster.
• Discussions wif other technical teams on regular basis regarding upgrades, Process changes, any special processing and feedback.

Environment: HDFS, Map Reduce Yarn, Hive, PIG, Sqoop, Oozie, Falcon, HBase, Zookeeper, Zeppelin, Ambari infra, Kafka, Kerberos and Ranger using Ambari.

Confidential

Hadoop Admin

Responsibilities:

• Configuring, Maintaining, and Monitoring Hadoop Cluster using Cloudera manager.
• Installed, configured and deployed a 20 node Cloudera Hadoop cluster for development.
• Commissioning and decommissioning of nodes on Hadoop cluster.
• Performed both major and minor upgrades to teh existing CDH cluster.
• Performance tune Hadoop cluster to achieve higher performance
• Configured Hive metastore wif MYSQL, which stores teh metadata of Hive tables
• Benchmarking Hadoop clusters using DFSIO, Teragen, and Terasort.
• Used Ganglia and Nagios for monitoring teh cluster around teh clock
• Moved data from HDFS to RDBMS and vice-versa using SQOOP.
• Used Hive and created Hive tables and involved in data loading.
• Used Hive to analyze teh partitioned and bucketed data and compute various metrics for reporting.
• Set up automated processes to analyze teh System and Hadoop log files for predefined errors and send alerts to appropriate groups.

Environment: HDFS, Map Reduce, Hive, Oozie, Pig, Flume, Sqoop, Kerberos, Zookeeper and Mysql using Cloudera Manager.

Confidential

System Admin

Responsibilities:

• Administration of RHEL, which includes installation, testing, tuning, upgrading and loading patches, troubleshooting both physical and virtual server issues.
• Creating, cloning Linux Virtual Machines.
• Installing Red Hat Linux using kick start and applying security polices for hardening teh server based on teh company policies.
• RPM and YUM package installations, patch and other server management.
• Managing systems routine backup, scheduling jobs like disabling and enabling cron jobs, enabling system logging, network logging of servers for maintenance, performance tuning, testing.
• Tech and non-tech refresh of Linux servers, which includes new hardware, OS, upgrade, application installation, testing.
• Set up user and group login ID's, printing parameters, network configuration, password, resolving permissions issues, and user and group quota.
• Installing MySqlDB in Linux and Customize teh MySQL DB parameters.
• Working wif Service Now incident tool.
• Creating physical volumes, volume groups and logical volumes.
• Samba Server configuration wif Samba Clients.
• Knowledge of IP tables, SELINUX.
• Modified existing Linux file systems to a Standard EXT3.
• Configuration and administration of NFS, FTP, SAMBA, NIS.
• Maintenance of DNS, DHCP and APACHE services on Linux machines.
• Gathering requirements from customers and business partners and design, implement and provide solutions in building teh environment.
• Installing and configuring Apache and supporting them on Linux production servers.

Environment: Red-Hat Linux Enterprise servers (HP Proliant DL 585, BL ... ML Series, SAN (Netapp), VERITAS Cluster Server 5.0, Windows 2003 server, Shell programming.