PROFESSIONAL SUMMARY:

• Over 6 years of professional IT experience which includes around 3 years of hands on experience in Hadoop using, Hortonworks, Cloudera and Hadoop working environment includes Map Reduce, HDFS, HBase, Zookeeper, Oozie, Hive, Sqoop, Pig, Spark and Flume.
• Hands on Experience in Installing, Configuring and using Hadoop Eco System Components like HDFS, Hadoop Map Reduce, Yarn, Zookeeper, Sqoop, NiFi, Flume, Hive, HBase, Spark, Oozie.
• Experience in deploying a Hadoop cluster using Hortonworks integrated with Ambari for monitoring and Alerting.
• Experience using Hortonworks platform and their eco systems. Hands on experience in installing, configuring and using ecosystem components like Hadoop MapReduce, HDFS, Hive and Flume.
• Installation, Configuration, and Administration of Hadoop cluster of major Hadoop distributions such as Hortonworks Data Platform (HDP1 and HDP2) and Cloudera Enterprise (CDH3 and CDH4)
• Experience in configuring various configuration files like core - site.xml, hdfs-site.xml, mapred-site.xml, yarn-site.xml based upon the job requirement.
• Experience in Apache Phoenix enables OLTP and operational analytics.
• Experience in installing and configuring the Zookeeper to co-ordinate the Hadoop daemons
• Working knowledge in importing and exporting data into HDFS using Sqoop.
• Strong experience in developing, debugging and tuning Map Reduce jobs in Hadoop environment.
• Experience in defining batch job flows with Oozie.
• Experience in Loading log data directly into HDFS using Flume.
• Experienced in managing and reviewing Hadoop log files to troubleshoot the issues occurred.
• Experience in following standard Back up Measures to make sure the high availability of cluster.
• Experience in Implementing Rack Awareness for data locality optimization.
• Experience in scheduling snapshots of volumes for backup and find root cause analysis of failures and documenting bugs and fixes, scheduled downtimes and maintenance of cluster.
• Good experience in Hive, Phoenix data modeling/queries.
• Experience in database imports, worked with imported data to populate tables in Hive.
• Hands on experience in data mining process, implementing complex business logic and optimizing the query using HiveQL and controlling the data distribution by partitioning and bucketing techniques to enhance performance.
• Experience working with Hive data, extending the Hive library using custom UDF's to query data in non-standard formats
• Exposure about how to export data from relational databases to Hadoop Distributed File System.
• Experience in cluster maintenance, commissioning and decommissioning the data nodes.
• Experience in monitoring systems and services, architecture design and implementation of Hadoop deployment, configuration management, backup, and disaster recovery systems and procedures.
• Experience in monitoring multiple Hadoop clusters environments using Cloudera Manager and Ambari as well as workload, job performance and capacity planning.
• Experience in installing and configuring Kerberos for the authentication of users and Hadoop daemons.
• Hands on experience in Linux admin activities on Cent OS.
• Knowledge on Cloud technologies like AWS Cloud.
• Experience in Benchmarking, Backup and Disaster Recovery of Name Node Metadata.
• Experience in performing minor and major Upgrades of Hadoop Cluster.
• Perform hands-on administration, monitoring and troubleshooting of all company networks and programs, resulting in optimum performance and minimum downtime.

TECHNICAL SKILLS:

Databases: MS SQL Server, Oracle, TeraData

Operating Systems: Windows Server … Windows 10/7/XP, Mac OS, Linux (Red Hat, Ubuntu, Cent OS 6.5)

Languages: Linux Commands, SQL Queries, UNIX Shell \Scripting, Core Java, Python

Cluster Management tools: Cloudera Manager and Ambari

Hadoop Frameworks: HDFS, Spark, Map Reduce, Hive, Pig, Zookeeper, Impala, Sentry, Hue, Yarn, Kafka

NoSQL: -Data Bases HBase

Data Ingestion / ETL tools: Flume, Sqoop, Kafka

WORK EXPERIENCE:

Confidential, Wilmington, DE

Hadoop Administrator

Responsibilities:

• Configuring, Maintaining, and Monitoring Hadoop Cluster using Ambari distribution.
• Day to day responsibilities includes solving developer issues, deployments moving code from one environment to other environment, providing access to new users and providing instant solutions to reduce the impact and documenting the same and preventing future issues.
• Responsible for Cluster maintenance, commissioning and decommissioning Data nodes, Cluster Monitoring, Troubleshooting, Manage and review data backups, Manage & review Hadoop log files.
• Enable High Availability Name Node, Resource manager, HBase and HiveServer2 automatic failover infrastructure to overcome single point of failure.
• Used NiFi for automation of data movement between desperate data sources and systems, making data ingestion fast, easy and secure.
• Installation of various Hadoop Ecosystems and Hadoop Daemons.
• Installed MySQL database to store metadata.
• Import and export data to hive tables and Hbase.
• Commissioning and Decommissioning Hadoop Cluster Nodes Including Balancing HDFS block data.
• Good experience in troubleshoot production level issues in the cluster and its functionality.
• Production jobs debugging when failed.
• Creating queues on YARN queue manager to share the resources of the Cluster for the MapReduce jobs given by the users.
• Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
• Responsible for creating Hive tables based on business requirements
• Experienced on adding/installation of new components and removal of them through Ambari.
• Monitored workload, job performance and capacity planning using Ambari.
• Participated in development and execution of system and disaster recovery processes.
• Loaded data into NoSQL database HBase
• Involved in extracting the data from various sources into Hadoop HDFS for processing.
• Inputs to development regarding the efficient utilization of resources like memory and CPU utilization based on the running statistics of Map and Reduce tasks.
• Changes to the configuration properties of the cluster based on volume of the data being processed and performance of the cluster.
• Discussions with other technical teams on regular basis regarding upgrades, Process changes, any special processing and feedback.
• Periodically reviewed Hadoop related logs and fixing errors and preventing errors by analyzing the warnings.

Environment: HDFS HDP, Ambari, Hbase, NOSQL, Python, Yarn, Hive, Kerberos, Pig, Hadoop, HDFS, Pig, Sqoop, HBase, NiFi, Shell Scripting, Linux Red Hat.

Confidential, Dearborn, MI

Hadoop Admin

Responsibilities:

• Responsible for building a cluster on HDP 2.3. With Hadoop 2.2.0 using Ambari.
• Responsible for implementation and ongoing administration of Hadoop administration.
• Involved in Performance testing of the Production Cluster using TERAGEN, TERASORT and TERAVALIDATE.
• Implemented commissioning and decommissioning of data nodes.
• Involved in Importing and exporting data into HDFS and Hive using Sqoop.
• Experienced in managing and reviewing Hadoop log files.
• Supported Map Reduce Programs those are running on the cluster.
• Involved in creating Hive tables, loading with data and writing hive queries, which will run internally in map.
• Managed 350+ Nodes HDP 2.3 cluster with 4 peta bytes of data using Ambari 2.0 and Linux Cent OS 7.
• Implemented Fair scheduler on the Resource Manager to allocate the fair amount of resources to small jobs.
• Installed and configured Hive Using Hive Metastore, Hiveserver2 and HCatalog.
• Created method of process for the Kerberos KDC cluster Setup.
• Tested configuring Kerberos KDC and Slave KDC and adding multiple realms to distinguish each Hadoop cluster.
• Having experience in creating policies in Ranger.
• Installed and Configured Hbase by installing Hbase Master and Hbase Regional Servers.
• Good experience in troubleshooting production level issues in the cluster and its functionality.
• Backed up data on regular basis to a remote cluster using distcp.
• Aligning with the systems engineering team to propose and deploy new hardware and software environments required for Hadoop and to expand existing environments.
• Collaborating with application teams to install operating system and Hadoop updates, patches, version upgrades when required.
• Experience in using NiFi for automation of data movement between desperate data sources and systems, making data ingestion fast, easy and secure.
• Rack Aware Configuration, Configuring Client Machines Configuring, Monitoring and Management Tools.
• Used Fair Scheduler to manage Map Reduce jobs so that each job gets roughly the same amount of CPU time.
• Handle the upgrades and Patch updates.
• Worked on configuring security for Hadoop Cluster, managing and scheduling jobs on a Hadoop Cluster.

Environment: HDFS Horton works HDP 2.3, Map Reduce, Hive, Pig Hbase, Sqoop, NiFi, RDBMS/DB used: Flat files, MySQL, Hbase, Ambari.

Confidential, Reston, VA

Hadoop Admin

Responsibilities:

• Gathered the business requirements from the Business Partners and Subject Matter Experts.
• Installed, Configured and Maintained the Hadoop cluster for application development and Hadoop ecosystem components like Hive, Pig, HDFS, Zookeeper and Sqoop.
• In depth understanding of Hadoop Architecture and various components such as HDFS, Name Node, Data Node, Resource Manager, Node Manager and YARN / Map Reduce programming paradigm.
• Monitoring Hadoop Cluster through Ambari and Implementing alerts based on Error messages. Providing reports to management on Cluster Usage Metrics and Charge Back customers on their Usage.
• Extensively worked on commissioning and decommissioning of cluster nodes, file system integrity checks and maintaining cluster data replication.
• Expertise on Cluster Planning, Performance tuning, Monitoring and Troubleshooting the Hadoop Cluster.
• Expertise on cluster audit findings and tuning configuration parameters.
• Expertise in configuring MySQL to store the hive metadata.
• Built high availability for major production cluster and designed automatic failover control using Zookeeper Failover Controller and Quorum Journal nodes.
• Extensively involved Commissioning and Decommissioning Nodes from time to time.
• Responsible on adding/installation of new services and removal of them through Ambari.
• Working with data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos principals and testing HDFS, Hive, Pig and MapReduce access for the new users.
• Periodically reviewed Hadoop related logs and fixing errors and preventing errors by analyzing the warnings.
• Responsible in setting log retention policies and setting up of trash interval time.
• Monitoring the data streaming between web sources and HDFS.
• Setting up HDFS Quotas to enforce the fair share of computing resources.
• Strong Knowledge in Configuring and maintaining YARN Schedulers (Fair, and Capacity)
• Wrote the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
• Involved in configuring Zookeeper to coordinate the servers in clusters to maintain the data consistency.
• Back up of data from active cluster to a backup cluster using DISTCP.
• Install operating system and Hadoop updates, patches, version upgrades when required.
• Dumped the data from HDFS to MYSQL database and vice-versa using Sqoop.

Environment: HDP Ambari, Cloudera Manager, HDFS, Unix/Linux, HDFS, MapReduce, YARN, Pig, Hive, HBase, Flume, Sqoop, NiFi, Shell Scripting, Ambari, Kerberos.

Confidential

Systems Administrator

Responsibilities:

• Manage entire company systems and infrastructure.
• Infrastructure consisting of several sites throughout Southern California and Hawaii.
• Lead support for network systems and Servers Company wide.
• Manage company remote sites. On site presence and remote connectivity for sites in other states.
• Manage Active Directory and Office 365.
• Manage ShoreTel VOIP System.
• Manage and provide application support for all Yardi ERP related issues.
• Provide an elevated level of hands-on support partnerships to corporate end users at the desktop level. Able to work both independently and collectively to find solutions to complex technical problems and to escalate more complex issues to senior IT team members.
• Participate in proactive preventative maintenance of Support Office and field IT systems to improve stability, systems' uptime and to prevent future problems.
• Work on special projects as needed to support the technical team.
• Project lead on configuring and database conversion with the company's self- storage accounting and customer database system. Worked with SQL to manipulate and convert the data from one database to another.
• Project lead on upgrading several company sites internet and phone connections. Upgraded internet speed, MPLS connectivity and converted PBX telephone system to VOIP telephone system.

Confidential

Systems Administrator

Responsibilities:

• Worked on Administration of RHEL 4.x and 5.x which includes installation, testing, tuning, upgrading and loading patches, troubleshooting both physical and virtual server issues.
• Created and cloned Linux Virtual Machines, templates using VMware Virtual Client 3.5 and migrated servers between ESX hosts and Xen servers.
• Installed RedHat Linux using kick-start and applying security polices for hardening the server based on the company policies.
• Installed RPM and YUM packages patch and another server management.
• Managed systems routine backup, scheduling jobs like disabling and enabling cron jobs, enabling system logging, network logging of servers for maintenance, performance tuning and testing.
• Worked and performed data-center operations including rack mounting and cabling.
• Set up user and group login ID, network configuration, password, resolving permissions issues, user and group quota.
• Configured multipath, adding SAN and creating physical volumes, volume groups, logical volumes.
• Manager, Samba, NFS, NIS, LVM, Linux, Shell Programming.
• Worked on daily basis on user access and permissions, Installations and Maintenance of Linux Servers.
• Installed Cent OS using Pre-Execution environment boot and Kick start method on multiple servers, remote installation of Linux using PXE boot.
• Monitored System activity, Performance and Resource utilization.
• Performed all System administration tasks like cron jobs, installing packages and patches.
• Used LVM extensively and created Volume Groups and Logical volumes.
• Performed RPM and YUM package installations, patch and another server management.
• Built, implemented and maintained system-level software packages such as OS, Clustering, disk, file management, backup, web applications, DNS, LDAP.
• Performed scheduled backup and necessary restoration.
• Configured Domain Name System (DNS) for hostname to IP resolution.
• Troubleshot and fixed the issues at User level, System level and Network level by using various tools and utilities.
• Schedule backup jobs by implementing cron job schedule during non-business hour.