OBJECTIVE

• To demonstrate my expertise as Hadoop Administrator to ensure Hadoop cluster administration and technical support in large scale IT industry

SUMMARY

• 7 plus years of professional experience including around 4 years of Unix Administrator and 3 plus years in Big Data analytics asHadoopAdministrator.
• Experience in all the phases of Data warehouse life cycle involving Requirement Analysis, Design, Coding, Testing, and Deployment.
• Experience in working with business analysts to identify study and understand requirements and translated them into ETL code in Requirement Analysis phase.
• Experience in architecting, designing, installation, configuration and management of ApacheHadoop Clusters & ClouderaHadoopDistribution.
• Experience in managing theHadoopinfrastructure with Cloudera Manager.
• Practical knowledge on functionalities of everyHadoopdaemons, interaction between them, resource utilizations and dynamic tuning to make cluster available and efficient.
• Experience in understanding and managingHadoopLog Files.
• Experience in understandingHadoopmultiple data processing engines such as interactive SQL, real time streaming, data science and batch processing to handle data stored in a single platform in Yarn.
• Experience in adding and removing the nodes onHadoopCluster.
• Experience in managing theHadoopcluster with IBM Big Insights, Horton Works.
• Experience in extracting the data from RDBMS into HDFS Sqoop.
• Experience in collecting the logs from log collector into HDFS using up Flume.
• Experience in setting up and managing the batch scheduler Oozie.
• Good understanding of No SQL databases such as HBase, Neo4j and Mongo DB.
• Experience in analyzing data in HDFS through Map Reduce, Hive and Pig.
• Design, implement and review features and enhancements to Cassandra.
• Deployed a Cassandra cluster in cloud environment as per the requirements.
• Experience on UNIX commands and Shell Scripting.
• Extensively worked on the ETL mappings, analysis and documentation of OLAP reports requirements.
• Experience in integration of various data sources like Oracle, DB2, Sybase, SQL server and MS access and non - relational sources like flat files into staging area.
• Experience in Data Analysis, Data Cleansing (Scrubbing), Data Validation
• Excellent interpersonal, communication, documentation and presentation skills.

TECHNICAL SKILLS

Hadoop Components: HDFS, Map Reduce, HBase, Pig,Hive

Linux Operating Systems: CENTOS, Debian, Fedora

Open Source Monitoring Tools: Nagios, Gangila, Cloudera

Networks: MVC, Spring, Hibernate Firewall, DNS

Administration Operations: Performance tuning, Storage capacity Management, System dump analysis

Programming Languages: Unix shell scripting, SQL, C, Java

Database: Oracle, DB2, MySQL

Database Tools: TOAD, Chordiant CRM tool, OWB

PROFESSIONAL EXPERIENCE

Hadoop Admin

Confidential, Chicago, IL

Environment: Hadoop, MapReduce, HDFS, Hive, Java, SQL, Cloudera Manager, Pig, Sqoop, Oozie

Responsibilities:

• Handle the installation and configuration of a Hadoop cluster.
• Build and maintain scalable data pipelines using the Hadoop ecosystem and other open source components like Hive and HBase.
• Handle the data exchange between HDFS and different Web Applications and databases using Flume and Sqoop.
• Monitor the data streaming between web sources and HDFS.
• Close monitoring and analysis of the Map Reduce job executions on cluster at task level.
• Inputs to development regarding the efficient utilization of resources like memory and CPU utilization based on the running statistics of Map and Reduce tasks.
• Changes to the configuration properties of the cluster based on volume of the data being processed and performance of the cluster.
• Setting up Identity, Authentication, and Authorization.
• Maintaining Cluster in order to remain healthy and in optimal working condition.
• Handle the upgrades and Patch updates.
• Worked extensively with Amazon Web Services and Created Amazon Elastic Map Reduce cluster in both 1.0.3 and 2.2.
• Managed data in Amazon S3, Implemented s3cmd to move data from clusters to S3.
• Set up automated processes to analyze the System andHadooplog files for predefined errors and send alerts to appropriate groups.
• Balancing HDFS manually to decrease network utilization and increase job performance.
• Commission and decommission the Data nodes from cluster in case of problems.
• Set up automated processes to archive/clean the unwanted data on the cluster, in particular on Name node and Secondary name node.
• Set up and manage High Availability Name node and Name node federation using Apache 2.0 to avoid single point of failures in large clusters.
• Discussions with other technical teams on regular basis regarding upgrades, Process changes, any special processing and feedback.

Hadoop Admin

Confidential, Minnesota, MN

Environment: Hadoop, MapReduce, HDFS, Hive, Oracle 11g, Java, Struts, Servlets, HTML, XML, SQL, J2EE, JUnit, Tomcat 6.

Responsibilities:

• Installed and configuredHadoopand responsible for maintaining cluster and managing and reviewing Hadooplog files.
• Worked with Horton works HDP2 including Pig, Hive, H Base.
• Installed & managing distributions ofHadoopCDH3, CDH4, Cloudera manager, Map R, Horton works.
• Load data from various data sources into HDFS using Flume.
• Worked on Cloudera to analyze data present on top of HDFS.
• Worked extensively on Hive and PIG.
• Worked on large sets of structured, semi-structured and unstructured data.
• Use of Sqoop to import and export data from HDFS to RDBMS and vice-versa.
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
• Good knowledge on reading data from Cassandra and also writing to it.
• Provided operational support services relating toHadoopinfrastructure and application installation.
• Handled the imports and exports of data onto HDFS using Flume and Sqoop.
• Supported technical team members in management and review ofHadooplog files and data backups.
• Participated in development and execution of system and disaster recovery processes.
• Formulated procedures for installation ofHadooppatches, updates and version upgrades.
• Automated processes for troubleshooting, resolution and tuning ofHadoopclusters.
• Set up automated processes to send notifications in case of any deviations from the predefined resource utilization.

Hadoop Admin

Confidential, NY

Environment: Hadoop, Apache Pig, Hive, OOZIE, SQOOP

Responsibilities:

• Installed/Configured/Maintained ApacheHadoopclusters for application development andHadoop tools like Hive, Pig, HBase, Zookeeper and Sqoop.
• Wrote the shell scripts to monitor the health check ofHadoopdaemon services and respond accordingly to any warning or failure conditions.
• Managing and scheduling Jobs on aHadoopcluster.
• DeployedHadoopCluster in the following modes.
• Pseudo-distributed, Fully Distributed.
• Implemented Name Node backup using NFS. This was done for High availability.
• Worked on importing and exporting data from Oracle and DB2 into HDFS and HIVE using Sqoop.
• Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS.
• Created Hive External tables and loaded the data in to tables and query data using HQL.
• Wrote shell scripts for rolling day-to-day processes and it is automated.
• Collected the logs data from web servers and integrated in to HDFS using Flume.
• Implemented Fair schedulers on the Job tracker to share the resources of the Cluster for the Map and Reduce jobs given by the users.

Linux Administrator

Confidential, Dallas,TX

Environment: Linux 5/4, Sun Solaris 10/9/, Oracle 10g, SUN Servers, SUN Fires, Linux, HP open view service desk (OVSD), Kickstart, Jump Start, Fujitsu Prime power servers, Samba, AutoSys, VERITAS Volume Manager (VVM), LDAP, EMC Storage SAN, VERITAS Cluster Server (VCS), VxVM, VMware servers, WebLogic, Jboss and Apache.

Responsibilities:

• Installing and upgrading OE & Red hat Linux and Solaris 8/ & SPARC on Servers like HP DL 380 G3, 4 and 5 & Dell Power Edge servers.
• Experience in LDOM's and Creating sparse root and whole root zones and administered the zones for Web, Application and Database servers and worked on SMF on Solaris 10.
• Experience working in AWS Cloud Environment like EC2 & EBS.
• Implemented and administered VMware ESX 3.5, 4.x for running the Windows, Centos, SUSE and Red hat Linux Servers on development and test servers.
• Installed and configured Apache on Linux and Solaris and configured Virtual hosts and applied SSL certificates.
• Implemented Jumpstart on Solaris and Kick Start for Red hat environments.
• Experience working with HP LVM and Red hat LVM.
• Experience in implementing P2P and P2V migrations.
• Involved in Installing and configuring Centos & SUSE 11 & 12 servers on HP x86 servers.
• Implemented HA using Red hat Cluster and VERITAS Cluster Server 5.0 for Web Logic agent.
• Managing DNS, NIS servers and troubleshooting the servers.
• Troubleshooting application issues on Apache web servers and also database servers running on Linux and Solaris.
• Experience in migrating Oracle, MYSQL data using Double take products.
• Experience working on LDAP user accounts and configuring ldap on client machines.
• Upgraded Clear-Case from 4.2 to 6.x running on Linux (Centos &Red hat)
• Worked on patch management tools like Sun Update Manager.
• Experience supporting middle ware servers running Apache, Tomcat and Java applications.
• Worked on day to day administration tasks and resolve tickets using Remedy.
• Used HP Service center and change management system for ticketing.
• Worked on the administration of the Web Logic 9, JBoss 4.2.2 servers including installation and deployments.
• Worked on F5 load balancers to load balance and reverse proxy Web Logic Servers.
• Shell scripting to automate the regular tasks like removing core files, taking backups of important files, file transfers among servers.

Linux/Systems Administrator

Confidential

Environment: Linux 5/4/3, Solaris 10/9/8, Solaris x86, SUSE, Sun Cluster 3.2, Windows 2003, Clariion CX200, Perl, WebLogic 8.1, Shell Scripting, Bash, VERITAS Volume Manager (VVM),VMWare ESX-3.5, BigIP, LDAP, Oracle 10g, IBM SVC, HP open view data protector, SAN storage, Perl, Shell Scripting, Remedy, VERITAS Cluster Server (VCS), VERITAS Netbackup 6.0, Oracle RAC, WebSphere 5x, JBoss 4.2/4.0

Responsibilities:

• Installing, configuring and updating Solaris 7, 8, Red Hat 7.x, 8, 9, Windows NT/2000 Systems using media and Jumpstart and Kick start.
• Installing and configuring Windows Active directory server 2000 and Citrix Servers.
• Published and administered applications via Citrix Meta Frame.
• Creating and Authenticating Windows user accounts on Citrix server.
• User administration which included creating backup account for new users and deleting account for the retired or deleted users.
• Managing Tape Drives and recycling it after specific period of time as per the firm's policies.
• Working with DBA's for writing Scripts to take database backup and scheduling backup using cron jobs.
• Creating UNIX and PERL scripts for automated data backup, status of the storage.
• Installing and configuring Oracle 8i database and Sybase server on Solaris after creating the file systems, users and tuning the kernel.
• Installed and Configured SSH Gate for Remote and Secured Connection.
• Setting up labs from scratch, testing hardware, installing and configuring various hardware devices like printers, scanners, modems, network and communication devices.
• Configuration of DHCP, DNS, NFS and auto mounter.
• Creating, troubleshooting and mounting NFS File systems on different OS platforms.