SUMMARY:

• Over all 8+ years of System Admin experience in IT background which includes 4+ years of experience in Hadoop Technologies.
• Experience in developing Database management, Administrating Linux, developing Map - reduce applications, designing, building and administrating large scale Hadoop production Clusters.
• Experience in deploying and managing the multi-node development, testing and production Hadoop cluster with different Hadoop components using Cloudera Manager and Hortonworks.
• Experience in working with Flume to load the log data from multiple sources directly into HDFS.
• Experience in big data technologies: Hadoop HDFS, Map-reduce, Pig, Hive, Oozie, Sqoop, Zookeeper and NoSQL.
• Experience in administering the Linux systems to deploy Hadoop cluster and monitoring the cluster.
• Experience in Big Data Hadoop Implementations.
• Experience on Commissioning, Decommissioning, Balancing, and Managing Nodes and tuning server for optimal performance of the cluster.
• As an admin involved in Cluster maintenance, trouble shooting, Monitoring and followed proper backup& Recovery strategies.
• Experience in Chef, Puppet or related tools for configuration management.
• Experience in HDFS data storage and support for running map-reduce jobs.
• Optimizing performance of Hbase/Hive/Pig jobs.
• Involved in Infrastructure set up and installation of Cloudera stack on Amazon Cloud.
• Excellent knowledge of in NOSQL databases like HBase, Cassandra. Experience in monitoring and troubleshooting issues with Linux memory, CPU, OS, storage and network.
• Knowledge on architecture and functionality of NOSQL DB like HBase, Cassandra and MongoDB.
• Experience in troubleshooting errors in HBase Shell/API, Pig, Hive, Sqoop, Flume, Spark and MapReduce.
• Experience with ingesting data from RDBMS sources like - Oracle, SQL and Teradata into HDFS using Sqoop.
• Experience in benchmarking, performing backup and disaster recovery of Name Node metadata and important sensitive data residing on cluster.
• Experienced in installation, configuration, supporting and monitoring 100+ node Hadoop cluster using Cloudera manager and Hortonworks distributions.
• Experience in designing and implementing HDFS access controls, directory and file permissions user authorization that facilitates stable, secure access for multiple users in a large multi-tenant cluster.
• Extensive experience in developing web, enterprise and SOA applications using Java, J2SE, MultiThreading, JavaBeans, JSP, Servlets, JNDI, JDBC, Hibernate, JPA, Spring, WebServices (SOAP and Restful), JSF, XSD, XML, XSLT, JSON, JAX-B, EJB, JMS, MQ-Series, HTML, Ajax, Oracle and Linux/UNIX.

PROFESSIONAL EXPERIENCE

Hadoop Administrator

Confidential -Edison, NJ

Responsibilities:

• Specifying the Cluster size, allocating Resource pool and monitoring of jobs.
• Configured the Hive set up.
• Export the result set from one SQL server to another MySQL using Sqoop.
• Helped in the HIVE queries for the analysts.
• Helped the team to increase Cluster from 25 Nodes to 40 Nodes. The configuration for additional Data Nodes was managed through Serengeti.
• Maintain System integrity of all sub-components across the multiple nodes in the cluster.
• Monitor Cluster health and clean up logs when required.
• Perform upgrades and configuration changes.
• Upgrading the Hadoop Cluster from CDH3 to CDH4 and setup High availability Cluster Integrate the HIVE with existing applications
• Configured MySQL Database to store Hive metadata.
• Involved in managing and reviewing Hadoop log files.
• Involved in running Hadoop streaming jobs to process terabytes of text data.
• Worked with Linux systems and MySQL database on a regular basis.
• Supported Map Reduce Programs those ran on the cluster.
• Involved in loading data from UNIX file system to HDFS.
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
• Commission/decommission Nodes as needed.
• Installed and configured Hive, Pig, Sqoop and Oozie on the HDP 2.2 cluster.
• Manage resources in a multi-tenancy environment.
• Configured the Zookeeper in setting up the HA Cluster
• Implemented Fair schedulers on the Job tracker to share the resources of the Cluster for the Map Reduce jobs given by the users.
• Set up the compression for different volumes in the cluster.
• Developed Map Reduce programs to perform analysis research, identify and recommend technical and operational improvements resulting in improved reliability efficiencies in developing the Cluster.
• Wrote some Map reduce jobs for benchmark tests and automated them in a script.
• Environment:HDFS, Hive, Pig, sentry, Kerberos, LDAP, YARN, Cloudera Manager, and Ambari

Hadoop Administrator

Confidential - Phoenix, AZ

Responsibilities:

• The specific System management tasks that were involved in this assignment were
• Build 20 Node Hadoop Cluster on Cloudera.
• Configured IPTABLES rules to allow the connection of application servers to the cluster and also setup NFS exports list and blocked unwanted ports.
• Upgraded the Cluster from CDH3U0 to CDH3U1. The tasks were first performed on the staging platform before doing it on production Cluster.
• Managed backups for key data stores
• Supported configuring, sizing, tuning and monitoring analytic clusters
• Implemented security and regulatory compliance measures
• Streamlined cluster scaling and configuration
• Monitoring cluster job performance and involved capacity planning
• Works with application teams to install operating system and Hadoop updates, patches,
• Version upgrades as required.
• Helped in the set-up of Map reduce jobs.
• Implemented Name Node backup using NFS. This was done for High availability.
• Configured Ganglia to send the required job metrics for chargeback.
• Wrote shell scripts for Key Hadoop services like zookeeper, warden down and also automated them t run by using CRON.
• Implemented Capacity schedulers on the Job tracker to share the resources of the Cluster for the Map Reduce jobs given by the users.
• Supported Data Analysts in running Map Reduce Programs.
• Worked on importing and exporting Data into HDFS and HIVE using Sqoop.
• Worked on analyzing Data with HIVE and PIG
• Helped in setting up Rack topology in the cluster.
• Helped in the day to day support for operations.

Hadoop Administrator

Confidential - Austin, TX

Responsibilities:

• Responsible for upgrading Horton works Hadoop HDP2.2.0 and Mapreduce 2.0 with YARN in Multi Clustered Node environment. Handled importing of data from various data sources, performed transformations using Hive, Map Reduce, Spark and loaded data into HDFS.
• Control wiring Troubleshooting Function generator Microsoft PowerPoint.
• Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Cluster Planning, Manage and review data backups, Manage & review log files
• Installed and configured Hortonworks and Cloudera distributions on single node clusters for POCs.
• Involved in implementing security on Hortonworks Hadoop Cluster using with Kerberos by working along with operations team to move non secured cluster to secured cluster.
• Experience in locating and marking pipelines, Experience in locating and marking pipelines.
• Provided/created relative documentation for Oracle Fusion Middleware 11g.
• Implementing a Continuous Delivery framework using Jenkins, Puppet, Maven& Nexus in Linux environment. Integration of Maven/Nexus, Jenkins, Urban Code Deploy with Patterns/Release, Git, Confluence, Jira and Cloud Foundry.
• Perform routine equipment maintenance and limited troubleshooting activities as required.
• Managing totally 2500 servers which comprises of SUN- Solaris, HP-UNIX, Linux and AIX.
• Played a key role in the development of middleware server side functionality of handling the requests and responses using Java Process Definition.
• Involved in implementing security on Hortonworks Hadoop Cluster using with Kerberos by working along with operations team to move non secured cluster to secured cluster.
• Experience with Database management systems like Oracle DB & IBM DB2; configuring & tuning the DBMS with the middleware systems.
• Performed the automation using Chef Configuration management and managing the infrastructure environment with Puppet.
• Hadoop security setup using MIT Kerberos, AD integration (LDAP) and Sentry authorization.
• Configured Static IP addresses for CentOS servers.
• Used JIRA to capture, organize and prioritize issues. Experience in partially administering JIRA for issue management
• Migrated services from a managed hosting environment to AWS including: service design, network layout, data migration, automation, monitoring, deployments and cutover, documentation, overall plan, cost analysis, and timeline.
• Work with Hadoop developers, designers in troubleshooting map reduce job failure issues and help the developers.
• Troubleshoot and Resolve middleware issues across development, testing and production environments.
• Managing Amazon Web Services (AWS) infrastructure with automation and configuration management tools such as Chef, Ansible, Puppet, or custom-built .designing cloud-hosted solutions, specific AWS product suite experience.
• Performed a Major upgrade in production environment from HDP 1.3 to HDP 2.2. As an admin followed standard Back up policies to make sure the high availability of cluster.
• Monitored multiple Hadoop clusters environments using Ganglia and Nagios. Monitored workload, job performance and capacity planning using Ambari.
• Environment: Hadoop, HDFS, Pig, Hive, MapReduce, Sqoop, HBase, DEVOPS, ANT, and Maven, Chef, Puppet, Devops, Jenkins.

Hadoop Administrator

Confidential - Chicago, IL

Responsibilities:

• Responsible to manage data coming from different sources and involved in HDFS maintenance and loading of structured and unstructured data.
• Processed Multiple Data sources input to same Reducer using Generic Writable and Multi Input format.
• Created Data Pipeline of Map Reduce programs using Chained Mappers.
• Visualize the HDFS data to customer using BI tool with the help of HIVE ODBC Driver.
• Familiarity with a NoSQL database such as MongoDB.
• Implemented Optimized join base by joining different data sets to get top claims based on state using Map Reduce.
• Worked big data processing of clinical and non-clinical data using MapR.
• Implemented complex map reduce programs to perform joins on the Map side using Distributed Cache in Java.
• Responsible for importing log files from various sources into HDFS using Flume.
• Created customized BI tool for manager team that perform Query analytics using HiveQL.
• Used Hive and Pig to generate BI reports.
• Imported data using Sqoop to load data from MySQL to HDFS on regular basis.
• Created Partitions, Buckets based on State to further process using Bucket based Hive joins.
• Created Hive Generic UDF's to process business logic that varies based on policy.
• Moved Relational Data base data using Sqoop into Hive Dynamic partition tables using staging tables.
• Worked on custom Pig Loaders and storage classes to work with variety of data formats such as JSON and XML file formats.
• Experienced with different kind of compression techniques like LZO, GZIP, and Snappy.
• Used Oozie workflow engine to manage interdependent Hadoop jobs and to automate several types of Hadoop jobs such as Java map-reduce Hive, Pig, and Sqoop.
• Developed Unit test cases using Junit, Easy Mock and MRUnit testing frameworks.
• Experienced in Monitoring Cluster using Cloudera manager.
• Environment:Hadoop, HDFS, HBase, MongoDB, MapReduce, Java, Hive, Pig, Sqoop, Flume, Oozie, Hue, SQL, ETL, Cloudera Manager, MySQL.

UNIX Admin/Engineer

Confidential

Responsibilities:

• Installed, configured and maintained DNS servers, Mail servers, FTP servers, NFS, SMB, NIS, NIS+ and Samba Web servers on Sun Solaris, and Linux platforms.
• Installed and configured Solaris 10 servers with Zones and containers on SUN M5000 and T2000 Servers with SUN ZFS
• Involved in setting up accounting systems and performing on going system with the administrative tasks. Extensive user and group management through distributed shell scripts on CLI and shared keys across servers.
• Configured and Administering Solaris Jumpstart Server, AIX NIM Server and Linux Kick start Server.
• Support Sun and Intel servers with Solaris and Linux operating systems
• Creating Sun virtualization zones/containers/LDOMS (logical domains)
• Mirrored volumes on the Sun Sparc servers using Veritas VxVM, SVM, ZFS
• Created NFS, UNIX admin mount point, and patch installation of all QA servers.
• Installed, Configured and supported Oracle HA resource groups
• Worked on CA NSM tool monitoring for monitoring various applications and networks.
• Installing and configuration and maintenance of Veritas Netback up 4.5/5.1 and TSM
• Implemented and configured Backup policies and Storage Units.
• Managing disk storage with Veritas Volume Manager 3.5/4.1 and Solaris Volume Manager with Veritas File System (VxFS)
• Written Shell Scripts to collect the System performance Information, account information on a daily basis.
• Implemented NIS and NFS for administrative and project requirements.
• Setting up the thresholds on the parameters (CPU/Memory/Disk/Network) and alerting the users when the usage reaches the thresholds.
• Involved in data canter migration, SAN switch and SAN storage migration.
• Monitored system resources, disk usage, scheduling backups and restore.
• Performed root cause analysis of technical issues.
• Volume Managers such as Solaris Volume Manager, ZFS, Veritas Volume manager on SAN storage such EMC Clariion, Symmetrix, etc.
• Troubleshooting of NFS servers, NFS Clients in Auto Mount Environment.
• Scripting for job automation using Shell and Perl scripting.
• Installed and configured Sun Cluster 3.0 & 3.1
• Solaris Volume Manager/Solstice Disk Suite for Disk Device Management.
• Handling CPU panic, memory problems and other hardware failures with coordination of vendors.
• Configured FTP, Telnet, FTP, SSH, ip tables and SUDO upgrades for the servers.
• Configured EMC/SAN disks in Solaris Servers and HP.
• Automatically and manually taking backups / archives, with previously configured policies using Veritas Netback up.

Linux Administrator

Confidential

Responsibilities:

• Worked with system, network, security and storage teams to prepare and configure new servers for the environment.
• Managed and resolved incident tickets opened by clients as well as those logged by event monitoring system.
• Managed and installed patches and software packages using YUM and RPM.
• Provided day to day support to IT applications and users group.
• Provided Technical supports for internal users and resolved trouble shooting tickets.
• Provided technical support for end-user customers, diagnosed and resolved problems associated with network connection.
• Worked in 24x7 environment for production support in an on-call rotation