SUMMARY

• Over 8 Years of extensive IT experience with 3 +years of experience as a Hadoop Administrator and 5 years of experience in UNIX/Linux Administrator along with SQL developer in designing and implementing Relational Database model as per business needs in different domains.
• Experience in installation, management and monitoring of Hadoop cluster using Cloudera Manager.
• Optimized the configurations of Map Reduce, pig and hive jobs for better performance.
• Backup configuration and Recovery from a Namenode failure.
• Expert level skills in Managing and Scheduling Jobs on a Hadoop cluster.
• Ability to think creatively to help design innovative solutions to complex analytical questions.
• Extensive experience in installation, configuration, management and deployment of Big Data components and the underlying infrastructure of Hadoop Cluster.
• Good working knowledge on importing and exporting data from different databases namely MySQL, PostgreSQL, Oracle into HDFS and Hive using Sqoop.
• Extensive experience in NoSQL and real time analytics.
• Strong knowledge on yarn terminology and the High - Availability Hadoop Clusters.
• Hands on experience in analyzing Log files for Hadoop and eco system services and finding root cause.
• Experience in Chef, Puppet or related tools for configuration management.
• As an admin involved in balancing the loads on server and tuning of server for optimal performance of the cluster.
• Expertise in Installing, Configuration and Managing Red hat Linux 5, 6.
• Good experience on scheduling cron jobs in Linux.
• Worked with PLSQL stored procedures in creating reports, which required modified data input from the source.
• Experience in encryption security layer in the Hadoop environment.
• Proactively maintain and develop all Linux infrastructure technology to maintain a 24x7x365 uptime service.
• Maintain best practices on managing systems and services across all environments.
• Fault finding, analysis and of logging information for reporting of performance exceptions.
• Proactively monitoring system performance and capacity planning.
• Manage, coordinate, and implement software upgrades, patches, hot fixes on servers, workstations, and network hardware.
• Create and modify scripts or applications to perform tasks.
• Provide input on ways to improve the stability, security, efficiency, and scalability of the environment.
• Install and maintain all server hardware and software systems and administer all server performance and ensure availability for same.
• Maintain all system backup and assist to restore all procedures and provide required training to all users.
• Design and maintain all system tools for all scripts and automation processes and monitor all capacity planning.
• Integrate all required software and resolve all issues across various technologies and design require enterprise servers and provide back up support.
• Evaluate all documents according to system requirements and evaluate all design and perform tests on all development activities and administer all complex methodologies.
• Develop an infrastructure to provide support to all business requirements and perform regular troubleshoot on system to resolve all issues.
• Monitor everyday systems and evaluate availability of all server resources and perform all activities for Linux servers.
• Assist to configure and deploy all virtual machines and install and provide backup to all configuration procedures.
• Maintain and monitor all patch releases and design various patch installation strategies and maintain all systems according to NIST standardization.
• Administer all performance for various resources and ensure optimization for same and provide support to all applications and ensure optimal level of customer services.
• Maintain and monitor all system frameworks and provide after call support to all systems and maintain optimal Linux knowledge.
• Perform troubleshoot on all tools and maintain multiple servers and provide back up for all files and script management servers.
• Perform tests on all new software and maintain patches for management services and perform audit on all security processes.
• Collaborate with other teams and team members to develop automation strategies and deployment processes.

TECHNICAL SKILLS

Hadoop Framework: HDFS, Map Reduce, Hive, Pig, Zookeeper, Sqoop, Hbase, Flume

OS: RedHat Linux, UNIX, Windows 2000/NT/XP

Languages: C, C++, SAS, PL/SQL

Scripting Languages: Unix Shell scripting

Database: Oracle 10g/11g, SQL server, Teradata

Database Tools: Oracle SQL Developer, SQL Plus

Version Control: CVS, SVN

PROFESSIONAL EXPERIENCE

Confidential, Charlotte, NC

Hadoop Administrator

Responsibilities:

• Involved in start to end process of Hadoop cluster setup where in installation, configuration and monitoring the Hadoop Cluster.
• Responsible for Cluster maintenance, commissioning and decommissioning Data nodes, Cluster Monitoring, Troubleshooting, Manage and review data backups, Manage & review Hadoop log files.
• Monitoring systems and services, architecture design and implementation of Hadoop deployment, configuration management, backup, and disaster recovery systems and procedures.
• Installation of various Hadoop Ecosystems and Hadoop Daemons.
• Responsible for Installation and configuration of Hive, Pig, HBase and sqoop on the Hadoop cluster.
• Configured various property files like core-site.xml, hdfs-site.xml, mapred-site.xml based upon the job requirement.
• Involved in loading data from UNIX file system to HDFS.
• Provisioning, installing, configuring, monitoring, and maintaining HDFS, Yarn, HBase, Flume, Sqoop, Pig, Hive.
• Expertise in recommending hardware configuration for Hadoop cluster.
• Installing, Upgrading and Managing Hadoop Cluster on Cloudera distribution.
• Trouble shooting many cloud related issues such as Data Node down, Network failure and data block missing.
• Managing and reviewing Hadoop and HBase log files.
• Experience with Unix or Linux, including shell scripting.
• Strong problem-solving skills.
• Worked on IBM Jupyter conductor tool, that helps users to run the codes.
• Loading the data from the different Data sources like (Teradata and DB2) into HDFS using sqoop and load into Hive tables, which are partitioned.
• Developed Hive UDF’s to bring all the customers information into a structured format.
• Developed bash scripts to bring the Tlog files from ftp server and then processing it to load into hive tables.
• Built automated set up for cluster monitoring and issue escalation process.
• Administration, installing, upgrading and managing distributions of Hadoop (CDH3, CDH4, Cloudera manager), Hive, HBase.

Environment: Hadoop, HDFS, Map Reduce, Shell Scripting, spark, Pig, Hive, HBase, Sqoop, Flume, Oozie, Zookeeper, cluster health, monitoring security, Red hat Linux, IBM jupyter conductor, impala, Cloudera Manager.

Confidential, Atlanta, GA

Hadoop Administrator

Responsibilities:

• Involved in design and planning phases of Hadoop Cluster planning.
• Responsible for Regular health checkups of the Hadoop cluster using custom scripts.
• Installed and configured multi-node fully distributed Hadoop cluster of large number of nodes.
• Provided Hadoop, OS, and Hardware optimizations.
• Installed and configured Cloudera Manager for easy management of existing Hadoop cluster.
• Monthly Linux server maintenance, shutting down essential Hadoop name node and data node.
• Collaborated with the infrastructure, network, database, application and BI teams to ensure data quality and availability.
• Involved in creating Hive tables, loading with data and writing hive queries that will run internally in map reduce way.
• Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
• Experienced in managing and reviewing the Hadoop log files.
• Balancing Hadoop cluster using balancer utilities to spread data across the cluster equally.
• Implemented data ingestion techniques like Pig and Hive on production environment.
• Routine cluster maintenance on every weekend to make required configuration changes, installation etc.
• Implemented Kerberos Security Authentication protocol for existing cluster.
• Designing and creating ETL jobs through Talend to load huge volumes of data into cassandra, Hadoop Ecosystems and relational databases.
• Worked extensively with sqoop for importing metadata from Oracle. Used Sqoop to import data from SQL server to Cassandra.
• Implement Flume, Spark, Spark Stream framework for real time data processing. Developed analytical components using Scala, Spark and Spark Stream. Implemented Proofs of Concept on Hadoop and Spark stack and different big data analytic tools, using Spark SQL as an alternative to Impala.
• Implemented Apache Spark data processing project to handle data from RDBMS and streaming sources.
• Monitoring and Debugging Hadoop jobs/Applications running in production.
• Worked on Providing User support and application support on Hadoop Infrastructure.
• Kerberos keytabs creation for ETL application use cases before on boarding to Hadoop.
• Responsible for adding User to Hadoop cluster.
• Worked on Evaluating, comparing different tools for test data management with Hadoop.
• Helped and directed testing team to get up to speed on Hadoop Application testing.

Environment: Java, RedHat Linux, HDFS, Map-Reduce, Hive, Pig, Sqoop, Spark, Scala, Flume, Zookeeper, Oozie, DB2, HBase and Pentaho.

Confidential, Chicago, IL

Hadoop Administrator

Responsibilities:

• Collaborating with application teams to install operating system and Hadoop updates, patches, version upgrades when required.
• Installed and implemented the monitoring tools like ganglia and Nagios on both the clusters.
• Build a new sandbox cluster for the testing purpose and move data from secure cluster to insecure sandbox cluster by using distcp.
• Performance tuning of Hadoop clusters and Hadoop MapReduce routines.
• Screen Hadoop cluster job performances and capacity planning.
• Monitor Hadoop cluster connectivity and security.
• Manage and review Hadoop log files.
• HDFS support and maintenance.
• Diligently teaming with the infrastructure, network, database, application and business intelligence teams to guarantee high data quality and availability.
• Installed and configured various components of Hadoop ecosystem and maintained their integrity.
• Planning for production cluster hardware and software installation on production cluster and communicating with multiple teams to get it done.
• Designed, configured and managed the backup and disaster recovery for HDFS data.
• Experience with Unix or Linux, including shell scripting.
• Installing, Upgrading and Managing Hadoop Cluster on Cloudera distribution.
• Commissioned Data Nodes when data grew and decommissioned when the hardware degraded.
• Worked with systems engineering team to plan and deploy new Hadoop environments and expand existing Hadoop clusters.
• Worked with application teams to install Hadoop updates, patches, version upgrades as required.
• Involved in implementing High Availability and automatic failover infrastructure to overcome single point of failure for Name node utilizing zookeeper services.
• Implemented HDFS snapshot feature.
• Involved in start to end process of Hadoop cluster setup where in installation, configuration and monitoring the Hadoop Cluster.
• Ran monthly security checks through UNIX and Linux environment and installed security patches required to maintain high security level for our clients.

Environment: Hadoop, Map Reduce, Shell Scripting, spark, Pig, Hive, Cloudera Manager, CDH 5.4.3, HDFS, Yarn, Hue, Sentry, Oozie, Zookeeper, Impala, Kerberos, cluster health, Puppet, Ganglia, Nagios, Flume, Sqoop, storm, Kafka, KMS

Confidential, Peachtree City, GA

UNIX Administrator

Responsibilities:

• Implemented and setup Virtualization environments for AIX LPARs, HP Integrity VMs; and Solaris Zones and Logical Domains
• Updated and created provisioning scripts to setup new operating systems and software for supported platforms
• Setup and maintained NFS/iSCSI, NetApp and Oracle ZFS Appliances
• Consolidated servers at numerous smaller remote data centers to three central data centers
• Managed Backups for critical data which was stored on more than 70 servers
• Stabilized system by disk replacement, firmware upgrade in SAN storage, Solaris Volume Management, clustering environment on scheduled maintenance hours.
• Enhanced business continuity procedure by adding critical middleware server and identified through power-down test activity.
• Resolved issues, planned requests as point-of-contact to vendors, oversaw developers, business users, following change control procedure, reported result Monitor everyday systems and evaluate availability of all server resources and perform all activities for Linux servers.
• Assist to configure and deploy all virtual machines and install and provide backup to all configuration procedures.
• Maintain and monitor all patch releases and design various patch installation strategies and maintain all systems according to NIST standardization.
• Administer all performance for various resources and ensure optimization for same and provide support to all applications and ensure optimal level of customer services.
• Maintain and monitor all system frameworks and provide after call support to all systems and maintain optimal Linux knowledge.
• Perform troubleshoot on all tools and maintain multiple servers and provide back up for all files and script management servers.
• Wrote and maintained shell scripts using Perl and Bash.
• Monitored, troubleshot, and resolved issues involving operating systems.
• Applied ITIL approach to incident and problem management.
• Developed and maintained troubleshooting journal for incident management team.
• Participated in on-call rotation to provide 24×7 technical support.
• Tested numerous software and hardware configurations during the development stages to recreate the operating environments utilized by customers in an effort to avoid the distribution of releases with bugs and/or erroneous documentation.
• Wrote utility scripts using BASH and KORN shell.
• Configured UNIX systems to use Active directory, KERBEROS, NTPD, XDMCP, LDAP, SSH, FTP, TFTP and DNS.
• Configured SUN Workstation and SUN Server.
• Performed problem diagnosis, corrected discrepancies, developed user and maintenance documentation, provided user assistance and evaluated system performance.
• Installed, configured third party applications, hardened new and existing servers and desktops.
• Converted all systems into trusted systems to comply with STIG.
• Assisted DISA's system security office with the security testing and evaluation process.

Environment: Solaris, HP UX with Oracle, Sybase, KERBEROS, monitoring tool HP OpenView ITO (OVO), Redhat Linux, Windows, FTP

Confidential

UNIX Administrator

Responsibilities:

• Responsible for scheduling and upgrading these servers throughout the year to the latest versions of software
• Communicated and worked with the individual application development groups, DBAs and the Operations
• Created custom monitoring plugins for Nagios using UNIX shell scripting, and Perl.
• Assisted developers with troubleshooting custom software, and services such as ActiveSync, CalDav, CardDav, and PHP
• Top level customer service and implementation for DKIM, SPF, and custom SSL/TLS security
• Implemented and performed initial configuration Nimble Storage CS460G-X2 array and migrated data from legacy BlueArc Titan storage array. Converted access from NFS to iSCSI
• Assigned to selected projects and successfully defined hardware and software needs to complete them.
• Recommended to a project leader for a new Sales Tax project to use repurposed servers, thus saving the project
• Supported and maintained over 250 AIX, HP-UX servers working with a team of eight administrators in a 24/7 data center
• Provided root cause analysis of incident reports during any downtime issues
• Provided customer with administrative support on a UNIX based platform historical query database serving 500+ users.
• Maintained SUN server hardware and performed basic troubleshooting on database problems and initiated necessary steps to fixing any found errors utilizing shell scripts.
• Served as Project lead on updating hardware and software for the backup schema on both Windows and UNIX based development networks.
• Troubleshot any errors found in code using simple PERL scripts.
• Planned and coordinated move of server equipment from older server area to the newer location then conducted setup.
• Documented troubleshooting guide for administrators to be used for on-call pager duty.
• Attended team meetings and handled light managerial duties in the absence of team lead.

Environment: Solaris, HP UX, Red Hat Linux, Windows, FTP, SFTP