SUMMARY:

• 7+ years of experience in IT Industry with 4+ years of strong experience in BIG DATA Hadoop Administration.
• Hands - on experience in designing and implementing solutions using Hadoop 2.4.0, HDFS 2.7, MapReduce 2,HBase 1.1, Hive 1.2, Oozie 4.2.0, Tez 0.7.0, Yarn 2.7.0, Sqoop 1.4.6, Solr, Zookeeper, MongoDB
• Having Knowledge to implement Hortonworks (HDP 2.1 and HDP 2.3, HDP 2.4 ), Cloudera ( CDH3, CDH4, CDH5 on Linux.
• Experience in Configuring Name-node High availability and Name-node Federation
• Experience in Disaster recovery and Backup activities
• Experience in Multi-node setup of Hadoop cluster
• Experience in Performance tuning and benchmarking of Hadoop Cluster
• Experience in Monitoring, maintenance and troubleshooting of Hadoop cluster.
• Experience in Security integration of Hadoop Cluster.
• Good knowledge on Kerberos Security.
• Setting up and integrating Hadoop eco system tools - HBase, Hive, Pig, Sqoop etc.
• Familiar with installing and configuring Solr 5.2.1 in Hadoop cluster and implementation of Solr collections.
• Familiar with writing Oozie workflows and Job Controllers for job automation - Hive automation.
• Experience in Importing and exporting data from different databases like MySQL, RDBMS into HDFS and HBASE using Sqoop.
• Strong knowledge in configuring High Availability for Name Node, HBase, Hive and Resource Manager
• Experience in deploying and managing the multi-node development and production Hadoop cluster with different Hadoop components (HIVE, PIG, SQOOP, OOZIE, FLUME, HCATALOG, HBASE, ZOOKEEPER) using Hortonworks Ambari.
• Hands on experience in installation, configuration, supporting and managing Hadoop Clusters using Horton works, Cloudera and Hoop Apache.
• Hadoop Cluster capacity planning, performance tuning, cluster Monitoring, Troubleshooting.
• Excellent command in creating Backups & Recovery and Disaster recovery procedures and Implementing BACKUP and RECOVERY strategies for off-line and on-line Backups.
• Involved in bench marking Hadoop/HBase cluster file systems various batch jobs and workloads
• Making Hadoop cluster ready for development team working on POCs.
• Experience in minor and major upgrades of Hadoop and Hadoop eco system
• Experience monitoring and troubleshooting issues with Linux memory, CPU, OS, storage and network
• Hands on experience in analyzing Log files for Hadoop and eco system services and finding root cause.
• Experience on Commissioning, Decommissioning, Balancing, and Managing Nodes and tuning server for optimal performance of the cluster.
• Experience in importing and exporting the data using Sqoop from HDFS to Relational Database systems/mainframe and vice-versa.
• Experience in importing and exporting the logs using Flume.
• Hands on experience in Linux admin activities on RHEL &Cent OS.
• Experience in deploying Hadoop 2.0 (YARN).
• Good knowledge on cluster monitoring tools like Ganglia and Nagios.
• In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce concepts
• Hands on experience on Unix/Linux environments, which included software installations/upgrades, shell scripting for job automation and other maintenance activities.
• Having Experience in developing Oozie workflows and Job Controllers for job automation - Hive automation and scheduling jobs in HUE Browser.
• Having Experience in developing Hive Queries and Hive query optimization by setting different queues.
• Experience in Importing and exporting data from different databases like MySQL, RDBMS into HDFS and HBASE using Sqoop.
• Experience in deploying and managing the multi-node development and production Hadoop cluster with different Hadoop components (HIVE, PIG, SQOOP, OOZIE, FLUME, HCATALOG, HBASE, ZOOKEEPER) using Hortonworks Ambari.
• Gaining optimum performance with data compression, region splits and by manually managing compaction in Hbase
• Having experience in Upgrading the HDP Cluster from HDP 2.1 to HPD 2.2 and then to HDP 2.3.
• Worked on upgrading cluster from HDP 2.3 to HDP 2.4
• Good knowledge on cluster monitoring tools like Ganglia and Nagios.
• Working experience in Map Reduce programming model and Hadoop Distributed File System.
• In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce concepts
• Hands on experience on Unix/Linux environments, which included software installations/upgrades, shell scripting for job automation and other maintenance activities
• Thorough knowledge and experience in SQL and PL/SQL concepts.
• Good Knowledge on database stored procedures, functions and Triggers.
• Self-starter and team player, capable of working independently and motivating a team of professionals.

TECHNICAL SKILLS:

Programming Languages: Core Java, C++

Distribution Frameworks: Hadoop

Hadoop Distributions: Hortonworks (HDP 2.3, 2.4 and HDP 2.5), Cloudera (CDH 4.7, 5.4)

Hadoop Technologies: MapReduce, Hbase 1.2.6, Hive1.2.2, Sqoop 1.4.4, Pig 0.12.1, Oozie 4.0.0, Zookeeper 3.4.7, Kafka 0.9.0, Flume 1.8.0

J2EE Components: Servlets, JSP

Frameworks: Hibernate

Operating Systems: Windows 7/8/10, Linux & Unix

RDBMS: Oracle 10g, 12c, MySQL

Scripting Languages: JavaScript

Markup Languages: HTML5

Web/Application Servers: Tomcat 7.0

IDE: Eclipse, IntelliJ IDEA

PROFESSIONAL EXPERIENCE:

Senior Big Data / Hadoop Administrator

Confidential, Mclean, VA

Responsibilities:

• Hands-on with Installation and configuration of Hortonworks Data Platform HDP 2.3.4
• Upgraded the HDP from 2.3.4 to 2.5 and 2.5.3.
• Worked on Hadoop Administration, responsibilities include software installation, configuration, software upgrades, backup and recovery, cluster setup, cluster performance and monitoring on daily basis, maintaining cluster up and run on healthy.
• Commissioning and Decommissioning of nodes which as issues and disk failures.
• Made partitions of the disk and mounted to the directories, implemented RAID10 to improve database performance.
• Kerberized the cluster and integrated with Active Directory/ LDAP
• Implemented the Blueprints to build the cluster in production and Lower Environment.
• Monitored cluster using ambari metrics and used Hortonworks Smart sense to improve the cluster performance.
• Implemented the security requirements for Hadoop and integrate with Kerberos authentication and authorization infrastructure.
• Designed, developed and implemented connectivity products that allow efficient exchange of data between the core database engine and the Hadoop ecosystem. Involved in defining job flows using Oozie for scheduling jobs to manage Apache Hadoop jobs.
• Worked on YARN capacity scheduler by creating queues to allocate resource guarantee to specific groups.
• Monitored and analyzed of the Map Reduce job executions on the cluster at the task level.
• Extensively involved in Cluster Capacity planning, Hardware planning, Performance tuning of the Hadoop Cluster.
• Installed Ambari on an already existing Hadoop cluster.
• Implemented Rack Awareness for data locality optimization.
• Optimized and tuned the Hadoop environments to meet performance requirements.
• Hand-On experience with AWS cloud with EC2, S3.
• Collaborating with offshore team.
• Ability to document existing processes and recommend improvements.
• Shares knowledge and assists other team members as needed.
• Assist with maintenance and troubleshooting of scheduled processes.
• Participated in development of system test plans and acceptance criteria.

Environment: Hortonworks HDP2.3x and HDP 2.4, Ambari, Sqoop, Hive, Mapreduce2, Zookeeper, Ambari blueprints, Yarn, Tez, Hortonworks Smart sense, Spark, Kafka.

Senior Hadoop Administrator

Confidential, St. Louis, MO

Responsibilities:

• Installed and configured 100+ node Hadoop clusters HDP 2.X.
• Designed the production cluster with AWS machines and configured the services as per the machine type and capacity.
• Implemented the AWS network and sub-network according to the VPC and AZ.
• Integrated Hadoop with SSL and enabled Kerberos for Authentication.
• Integrated all clusters with the global realm.
• Upgrade cluster from HDP 2.3 to HDP 2.4.
• Worked with DevOps team to integrate and automate Hadoop configuration through the puppet.
• Added 100+ new nodes to production clusters.
• Active role in designing and building Lower Environment for the production has Test cluster, Development cluster, Acceptance cluster and Performance cluster image to the Production.
• Implemented the Blueprints to build the single and multi-node cluster in production and Lower Environment.
• Installed HDP 2.3 cluster with High available Name node and Resource manager.
• Applied patches and bug fixes on Hadoop services to overcome the Issues.
• Installed HST (Hortonworks Support Tool) and configured.
• Involved in Hadoop cluster auditing for standardization.
• Worked on Providing User support and application support on Hadoop Infrastructure.
• Performance tuned and optimized Hadoop clusters to achieve high performance.
• Monitoring Hadoop Clusters and supported 24x7 on-call Platform Support.
• Expertise in implementation and designing of disaster recovery plan for Hadoop Cluster.
• Extensive hands-on experience in Hadoop file system commands for file handling operations.

Environment: Horton work distribution HDP 2.4, HDFS, HA, Kafka 0.8, Storm 1.0, HBase 1.1.4, Rest, Knox, Kerberos, MapReduce, Yarn, Hive 1.0.1, Flume 1.4, AWS, Shell Script.

Hadoop Administrator

Confidential, NYC, NY

Responsibilities:

• Design/Implement/Architect Hadoop Ecosystem for Production/Disaster Recovery.
• Design/Implement automated deployments using CHEF.
• Installed and configured Hadoop clusters across various environments.
• Performed both major and minor upgrades to the existing Cloudera Hadoop cluster.
• Installed 200+ node Hadoop clusters using Cloudera Hadoop (CDH5.X)
• Applied patches and bug fixes for custom Hadoop software.
• Design/Implement Hadoop Schedulers for resource management.
• Design and Implement High Availability for all Hadoop Ecosystem components including NN, RM, HBase, Hive, Hive Server2, Oozie, Hue.
• Integrate Hadoop clusters with AD/Microsoft Kerberos with MIT Kerberos for Access Controls.
• Configure Cross Realm trust between environments for Data backup/DR.
• Expertise in implementation and designing of disaster recovery for Hadoop Cluster.
• Extensive hands-on experience in Hadoop file system commands for file handling operations.
• Back up / Monitor and configure alerts for Hadoop Ecosystem components using Cloudera Manager, Ganglia, Nagios, Custom scripts.
• Worked on Providing 400+ User base support and application support on Hadoop Infrastructure.
• Reviewing ETL application use cases before on boarding to Hadoop.
• Created Hive external tables and managed tables, designed data models in the hive.
• Involved in business requirements gathering and analysis of business use cases.
• Prepared System Design document with all functional implementations.
• Involved in Data model sessions to develop models for HIVE tables.
• Understanding the existing Enterprise data warehouse set up and provided design and architecture suggestion converting to Hadoop using MR, HIVE, SQOOP and Pig Latin.
• Worked on Sequence files, bucketing, partitioning for hive performance enhancement and storage improvement.
• Worked with SQOOP import and export functionalities to handle large data set transfer between DB2 database and HDFS.

Environment: Cloudera distribution CDH 5, HDFS, HA, Strom 1.0, Kerberos, MapReduce, Yarn, Hive 0.9, Flume 1.2.1, Sqoop 1.4, Oozie 3.3, Gangalia, Nagios, Shell Script, Perl.

Hadoop Administrator

Confidential, Wayne, PA

Responsibilities:

• Worked in Huge Cluster on maintaining over 400 nodes with in High availability environment.
• Worked on Hortonworks Distribution on major contributors to Apache Hadoop.
• Installation and configuration, Hadoop Cluster and Maintenance, Cluster Monitoring and Troubleshooting and Transform data from RDBMS to HDFS.
• Responsible for implementation and ongoing administration of Hadoop infrastructure.
• Using Hadoop cluster as a staging environment for the data from heterogeneous sources in data import process.
• Involved in Hadoop Cluster environment that includes adding and removing cluster nodes, cluster capacity planning, performance tuning, cluster monitoring, Troubleshooting.
• Configured High Availability on the name node for the Hadoop cluster - part of the disaster recovery roadmap.
• Adding new Nodes to an existing cluster, recovering from a Name Node failure.
• Decommissioning and commissioning the Node on running cluster.
• Scripting Hadoop package installation and configuration to support fully automated deployments.
• Developed Pig Scripts, Pig UDFs and Hive Scripts, Hive UDFs for analyzing the data loaded in Hadoop from various data sources.
• Used Sqoop to import data into HDFS from MySQL and Access databases and vice-versa.
• Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop.
• Involved working in Database backup and recovery, Database connectivity and security.
• Performed data completeness, correctness, data transformation and data quality testing using Instrumental in building scalable distributed data solutions using Hadoop eco-system.
• Developed several advanced Map Reduce programs to process data files received.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
• Experience in providing support to data analyst in running Pig and Hive queries.
• Managing and reviewing Hadoop log files.
• Creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
• Installing and configuring Hive and also written Hive UDFs experience in large scale data processing, on an Amazon EMR cluster.
• Efficient to handled Hadoop admin and user command for administration.
• Supported technical team members for automation, installation and configuration tasks.
• Wrote shell scripts to monitor the health check of Hadoop Components services and respond accordingly to any warning or failure conditions.
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
• Store unstructured data in semi structure on HDFS using HBase.
• Used Change management and Incident management process following the company standards.
• Implemented partitioning, dynamic partitions and buckets in HIVE.
• Installed, configured and maintained Hadoop cluster.
• Implemented Oozie workflows for Map Reduce, Hive and Sqoop actions.
• Implemented new feature Smart sence in 2.3 HDP.
• Involved in data migration from Oracle database to MongoDB.
• Created HBase tables to store variable data formats of data coming from different applications.
• Experience in managing and reviewing Hadoop log files.
• Involved in moving all log files generated from various sources to HDFS for further processing through Flume.
• Channelized Map Reduce outputs based on requirement using Partitioners.
• Involved in estimation and setting-up Hadoop Cluster in Linux.
• Wrote custom scripts in Shell to back up metadata periodically. Have been able to troubleshoot the cluster and configure the required properties in HDFS.
• Coordinating with system owners, database team, storage team, network team, hardware team, and other teams as when required to resolve issues.
• Providing root cause analysis for system-related failures and coordinating with required teams.
• Performing planned and Break fix changes in Infrastructure.

Environment: Cent OS, CDH 5.4.5, Oracle, MS-SQL, Zookeeper3.4.6, Oozie 4.1.0, MapReduce, YARN 2.6.1, Nagios, REST APIs, Amazon web services, HDFS, Sqoop1.4.6, Hive 1.2.1, Pig 0.15.0

Linux System Administrator

Confidential

Responsibilities:

• Designed Integrate Screens with Java Swings for displaying the transactions.
• Involved in the development of code for connecting to the database using JDBC with the help of Oracle Developer 9i.
• Involved in the development of database coding including Procedures, Triggers in Oracle.
• Worked as Research Assistant and a Development Team Member.
• Coordinated with Business Analysts to gather the requirement and prepare data flow diagrams and technical documents.
• Identified Use Cases and generated Class, Sequence and State diagrams using UML.
• Used JMS for the asynchronous exchange of critical business data and events among J2EE components and the legacy system.
• Worked in Designing, coding, and maintaining of Entity Beans and Session Beans using EJB 2.1 Specification.
• Worked in the development of Web Interface using MVC Struts Framework.
• User Interface was developed using JSP and tags, CSS, HTML, and JavaScript.
• Database connection was made using properties files.
• Used Session Filter for implementing timeout for ideal users.
• Used Stored Procedure to interact with the database.
• Development of Persistence was done using DAO and Hibernate Framework.
• Used Log4j for logging.

Environment: Red hat Linux/CentOS4, 5, Logical Volume Manager, Hadoop, VMware ESX 3.0, Kernel and resource tuning, Apache and Tomcat Web Server, Oracle 9g, Oracle RAC, HPSM, HPSA

Linux System Administrator

Confidential

Responsibilities:

• Administration, package installation, configuration of Oracle Enterprise Linux 5.x.
• Administration of RHEL, which includes installation, testing, tuning, upgrading and loading patches, troubleshooting both physical and virtual server issues.
• Creating, cloning Linux Virtual Machines.
• Installing RedHat Linux using kick start and applying security polices for hardening the server based on the company policies.
• RPM and YUM package installations, patch and other server management.
• Managing systems routine backup, scheduling jobs like disabling and enabling cron jobs, enabling system logging, network logging of servers for maintenance, performance tuning, testing.
• Tech and non-tech refresh of Linux servers, which includes new hardware, OS, upgrade, application installation, testing.
• Set up user and group login ID's, printing parameters, network configuration, password, resolving permissions issues, and user and group quota.
• Creating physical volumes, volume groups, and logical volumes.
• Gathering requirements from customers and business partners and design, implement and provide solutions in building the environment.
• Installing and configuring Apache and supporting them on Linux production servers.

Environment: Solaris, Red hat, VERITAS Volume Manager, cron jobs, Sun and VERITAS Clusters, Shell scripting.