Hadoop Administrator/developer Resume
Naperville, IL
PROFESSIONAL SUMMARY:
- Around 8 years of Strong Experience in LINUX/UNIX Administration, Shell Scripting with expertise in Red - hat Linux 5 and 6.x,Solaris 8, 9 and 10. HP-UX 11.i, AIX 5.1, 5.3 and 6.1, Windows 2003
- Around 4 years of Strong Experience in HADOOP Administration, including Tools like HIVE, PIG, SQOOP, FLUME, IMPALA, ZOOKEEPER and related BIG DATA Tools with a total of 8+ years experience in UNIX/LINUX Systems administration with expertise in Red-hat Linux 5 and 6.x, Solaris 8, 9 and 10. HP-UX 11.i, AIX 5.1, 5.3 and 6.1, Windows 2003, Web Logic 10.x, Web Sphere JAVA, J2EE Applications and SQL.
- Strong Experience in Installation and configuration of cloudera distribution Hadoop CDH4,CDH5.x, Namenode, secondary Namenode, jobtracker, and datanodes.
- Strong Experience in Installation and configuration of Hadoop ecosystem like Hbase, flume, Pig, sqoop.
- Expertise in Hadoop cluster task like Adding Nodes, Removing Nodes without any effect to running jobs and data.
- Strong in troubleshooting, monitoring, training and 24/7 production support in mission critical environments Worked with systems support team to define, architect and maintain highly available production systems
- Expertise in Installation, Configuration, Integration, Tuning, Backup, Crash recovery, Upgrades, Patching, Monitoring System Performance, System and Network Security and Troubleshooting of Unix Servers.
- Experience in implementing and monitoring High Availability environment using VERITAS Cluster Server, Sun Cluster, and Red hat Cluster, HP Service Guard and HACMP.
- Configured and used Jumpstart, Kick start, Ignite and NIM to install OS.
- Worked on Solaris 10 Container Technology. Created and maintained local zones.
- Managed resources for zones using Solaris 10 container technology.
- Solaris Live Update Technology associated with an operating system upgrade or maintenance.
- Strong experience in VERITAS Clusters in supporting high availability mission critical servers.
- Experienced with the installation, configuration and volume/File system management using VERITAS Volume Manager (VxVM), Logical Volume Manager (LVM) and Solstice Disk Suite (Solaris Volume Manager).
- Backup management and recovery policies using VERITAS Net-Backup.
- Experience in creating physical volume, logical volume, volume groups using LVMs in Linux/AIX, HP-UX (ITANIUM and PA-RISC) and SVM, GFS(global file system) and ZFS in Solaris.
- Proficient in Networking, configuring TCP/IP, DNS, NFS, NIS, NIS+, SAMBA, LDAP, SSH, SSL, SFTP, SMTP, SNMP.
- Experience in VMware Installed and monitored Virtual environments with ESX 3.x servers and Virtual Center 2.X
- Experienced in working with Storage Area Network (SAN), EMC CLARiiON CX, EMC Symmetrix DMX, EMC MirrorView, HP storage and Hitachi data storage.
- Expert in Installation, configuration of Patches and Packages.
- Worked on Puppet, HP Opsware, Nagios, Nimbus, Spectrum, Cacti monitoring, and configuration management softwares.
- Experience in installation, configuration and administration of Web Logic and Web sphere, Web sphere MQ Servers.
- Experience in writing Shell/Perl Scripts.
- Technical professional with management skills, excellent business understanding and strong communication skills.
- Good at Project management & Configuration management
TECHNICAL SKILLS
- HDFS
- Map
- HBase
- Red
- CENTOS
- Debian
- Fedora
- Nagios
- Ganglia
- Cloudera Manager
- TCP/IP
- Firewall
- DNS
- Unix shell scripting
- SQL
- C
- JavaOracle
- DB2
- MySQL
- SVN
- VSS
PROFESSIONAL EXPERIENCE
Confidential, Naperville IL
Hadoop Administrator/Developer
Responsibilities:
- Involved as a Hadoop Cluster environment administrator.
- Cluster capacity planning, performance tuning, cluster Monitoring, Troubleshooting.
- Configured Pseudo-distributed and Fully-distributed Hadoop clusters.
- Using Network Monitoring Daemons like Ganglia and Service monitoring tools like Nagios.
- Decommissioning and commissioning the Nodes on running cluster.
- Installation of Hadoop Ecosystems and Hadoop Daemons.
- Installation and configuration of Sqoop and Flume.
- Configured various property files like core-site.xml, hdfs-site.xml, mapred-site.xml based upon the requirement.
- Cluster monitoring, maintenance and troubleshooting;
- Importing and exporting data into HDFS using Sqoop.
- Experienced in defining job flows with Oozie.
- Loading log data directly into HDFS using Flume.
- Experienced in managing and reviewing Hadoop log files.
- Setup alerts with Cloudera Manager about memory and disk usage on the cluster.
- Used HIVE to aggregate the raw data that we get from clients.
- Involved with Kerberos Security on Hadoop Clusters.
- Assisted in designing, development and architecture of Hadoop and HBase systems.
- Coordinated with technical teams for installation of Hadoop and third related applications on systems.
- Formulated procedures for planning and execution of system upgrades for all existing Hadoop clusters.
- Supported technical team members for automation, installation and configuration tasks.
- Suggested improvement processes for all process automation scripts and tasks.
- Provided technical assistance for configuration, administration and monitoring of Hadoop clusters.
- Conducted detailed analysis of system and application architecture components as per functional requirements.
- Participated in evaluation and selection of new technologies to support system efficiency.
- Take ad-hoc requests from the developers to get the non-PII (Personally Identifiable Information) from the production cluster to non-Prod cluster to test out various scenarios.
- Installation and administration of RHEL 4.0/5.0, Solaris 9/10, AIX 5.3/6.1 and Suse 10.x.
- Configured kick start server and updating/applying patches to the servers using Red hat Satellite server.
- Planning and implementing system upgrades including hardware, operating system and periodical patch upgrades.
- Applied appropriate support packages/patches to maintain system integrity.
- Performed capacity analysis, monitored and controlled disk space usage on systems.
- Monitored system activities and fine-tuned system parameters and configurations to optimize performance and ensure security of systems.
- Responsible for architecting Hadoop clusters with CDH3.
- Experience in Installation and configuration of cloudera distribution Hadoop 2, 3, Namenode, secondarynamenode, jobtracker, Task trackers and data nodes.
- Strong Experience in Installation and configuration of Hadoop ecosystem like Hbase, flume, Pig, sqoop.
- Expertise in Hadoop cluster task like Adding Nodes, Removing Nodes without any effect to running jobs and data.
- Install, Configure and Manage of Hadoop Cluster spanning multiple racks using automated tools like puppet
- Perform Infrastructure services (DCHP, PXE, DNS, KICKSTART, and NFS).
- Support data analysis projects using Elastic Map Reduce on the Amazon Web Services (AWS) cloud.
- Provide Technical support for the Research in Information Technology program
- Manage and upgrade Linux and OS X server systems.
- Install and upgrade Plone, Zope and Apache web servers.
- Responsible for installation, configurations and management for Linux Systems
- Jumpstarting Solaris 9, 10 on Sun servers, configuring servers. Configured/Maintained Zones, LDOMs, ZFS file system Sun Solaris 10 Servers.
- Management and monitoring of various services using Service Management Facility and Service Administration.
- Installing, configuring VERITAS Netback up 6.5 on Linux, Solaris servers, creating backup polices.
- Designing Firewall rules for new servers to enable communication with application, Oracle 10g servers.
- Working on Volume management, Disk Management, software RAID solutions using VERITAS Volume manager & Solaris Volume Manager. File system Tuning and growing using VERITAS File System (VxFS), coordinated with SAN Team for storage allocation and Disk Dynamic Multi path.
- Worked on resolving production issues and documenting Root Cause Analysis and updating the tickets using BMC Remedy.
- Configuring, adding Oracle ASM disks to database server. Creating, managing raw volumes using LVM.
- Organized various meetings with teams as mentioned above for tracking their requirements as inputs to get this done and designed the plan/steps involved and took user approvals.
- Setting up network environments using TCP/IP, NIS, NFS, DNS, SNMP agents, DHCP and Proxy.
- Installation, maintenance and administration of Oracle and Db2 Server on Sun Servers.
Environment: Solaris 2.9 / 2.10, Red Hat Linux AS/EL 4.0, AIX 5.2, 5.3, Sun E10k, E25K, E4500, SunFire V440/880, DMX 3 & DMX4, CDH3,clouderamanager,Hbase,flume,pig, JDK 1.5,1.6 21,24,VxFS 4.1, VxVM 4.1, SVM.
Confidential, Niles IL
Hadoop Administrator
Responsibilities:
- Assisted in designing, development and architecture of Hadoop and HBase systems.
- Coordinated with technical teams for installation of Hadoop and third related applications on systems.
- Formulated procedures for planning and execution of system upgrades for all existing Hadoop clusters.
- Supported technical team members for automation, installation and configuration tasks.
- Suggested improvement processes for all process automation scripts and tasks.
- Provided technical assistance for configuration, administration and monitoring of Hadoop clusters.
- Conducted detailed analysis of system and application architecture components as per functional requirements.
- Participated in evaluation and selection of new technologies to support system efficiency.
- Assisted in creation of ETL processes for transformation of data sources from existing RDBMS systems.
- Designed and developed scalable and custom Hadoop solutions as per dynamic data needs.
- Coordinated with technical team for production deployment of software applications for maintenance.
- Assisted in creation of ETL processes for transformation of data sources from existing RDBMS systems.
- Designed and developed scalable and custom Hadoop solutions as per dynamic data needs.
- Coordinated with technical team for production deployment of software applications for maintenance.
- Provided operational support services relating to Hadoop infrastructure and application installation.
- Supported technical team members in management and review of Hadoop log files and data backups.
- Participated in development and execution of system and disaster recovery processes.
- Formulated procedures for installation of Hadoop patches, updates and version upgrades.
- Automated processes for troubleshooting, resolution and tuning of Hadoop clusters.
- Provided operational support services relating to Hadoop infrastructure and application installation.
- Supported technical team members in management and review of Hadoop log files and data backups.
- Participated in development and execution of system and disaster recovery processes.
- Formulated procedures for installation of Hadoop patches, updates and version upgrades.
- Automated processes for troubleshooting, resolution and tuning of Hadoop clusters. Responsible for architecting Hadoop clusters with CDH3.
- Experience in Installation and configuration of cloudera distribution Hadoop 2, 3, Namenode, secondarynamenode, jobtracker, Task trackers and data nodes.
- Strong Experience in Installation and configuration of Hadoop ecosystem like Hbase, flume, Pig, sqoop, zookeeper.
- Expertise in Hadoop cluster task like Adding Nodes, Removing Nodes without any effect to running jobs and data.
- Install, Configure and Manage of Hadoop Cluster spanning multiple racks using automated tools like puppet
- Responsible for installation and configuration of RHEL, Solaris, SUSE, Centos servers using kick start and jumpstart.
- Responsible for installation, configuration, up gradation of cloudera manager.
- Migration of various beta versions of CDH3B1, 2, 3 to CDH3U4.
- Configuring the racktopolgy script for the placing the blocks on different racks for more redundant data.
- Monitoring the Hadoop cluster and its ecosystem using tools like Nagios and Gangilia.
- Performance tuning of the cluster by tweaking the configuration parameters on hdfs-site.xml, mapred-site.xml, core-site.xml.
- Setting up automated scripts to check the integrity of the blocks on datanodes using Hdfsck and for running the balancer script to review the placements of blocks, ensuring all the datanodes are on under equal utilization.
- Troubleshooting and resolving the problems for the tickets raised by the customers.
- Worked in Multiple Environment Build, DEV, QA, UAT, PROD, and DR in installation, configuration and support.
- Responsible for 24/7 support and participated in on call support rotation.
- Working and assisting the developers for the successful deployment of the Map reduce jobs.
- Updating the Project managers on the status and my accomplishments on a weekly basis.
- Monitoring the web GUI and also the logs while the deployment of map reduces jobs to ensure mappers and reducers are not struck in an infinite loop.
Environment: RHEL 5.x,6.x, Red-Hat Linux Enterprise servers (HP Proliant DL 585, BL 465/485,ML Series), Solaris 8,9,10, AIX 5.3, 6.1, SAN(Netapp), Blade Logic, VERITAS Cluster Server 5.0, Windows 2003 server, Shell programming, Jboss 4.2, JDK 1.5,1.6 21,24,, VMware Virtual Client 3.5, VMware Infrastructure 3.5 CDH2,CDH3,cloudera manager, Hbase, Zookeeper.
Confidential, Atlanta,GA
UNIX Administrator
Responsibilities:
- Installation, Maintenance, Administration and troubleshooting of OEL, Linux, Sun Solaris, AIX, HP-UX systems.
- Performed automated installations of Operating System using Jumpstart for Solaris and kick start for Linux.
- As a Solaris administrator primary responsibility includes building of new servers which includes rack mounting, installation of OS, configuring of various OS-native and third party tools, securing of OS, installing & configuring, job scheduling.
- Installation and configured Solaris 10 with Zones, Containers and Ldoms.
- Extensively worked on hard disk mirroring and stripe with parity using RAID controllers.
- Involved in Server sizing and identifying, recommending optimal server hardware based on User requirements.
- Installation and configuration of VERITAS Storage Foundation Suite::VERITAS Volume Manager, VERITAS File System, VERITAS Cluster Server and Oracle SF RAC.
- Installed the Operating System on Solaris and Linux servers and Blades over the network.
- Built servers according to standards specified as per individual applications
- Built virtualized Linux servers on ESX and Blade servers to host multiple applications on same chassis across different server hosts
- Set up Linux servers for iSCSI LUN boot instead of using local hard disks
- Set up Sun servers for SAN-through-FCP boot as against booting from internal hard disks
- Configured Sun servers M4000 and M5000 for operational readiness upon OS installation
- Set up Linux servers for LAMP implementations and readiness
- Created File systems from local storage as well as NFS partitions for application portability
- Created Zeta byte file system (ZFS) in Solaris 10. Created zpools. Exported ZFS from local zones to local zones.
- Build LPAR system and Installed AIX 5.3 and 6.1 on LPAR using NIM Power 4, 5 and 6 systems and Upgraded Maintenance level as well.
- Installed and verified that all AIX/Linux patches or updates are applied to the servers.
- Configured HACMP 4.5 to monitor, detect and react to failure events, allowing the system to stay available during random, unexpected software problems and also configured to react to hundreds of system events.
- Installing, administering Red hat using Xen, Oracle Virtual Machine based hypervisors.
- RPM and YUM package installations, patch and other server management.
- Managing systems routine backup, scheduling jobs like disabling and enabling cron jobs, enabling system logging, network logging of servers for maintenance, performance tuning, testing.
- Worked and performed data-center operations including rack mounting, cabling.
- Installed, configured, and maintained Web Logic 10.x and Oracle 10g on Solaris & Red hat Linux.
- Configuring multipath, adding SAN and creating physical volumes, volume groups, logical volumes.
- Installing and configuring Apache and supporting them on Linux production servers.
- Troubleshooting Linux network, security related issues, capturing packets using tools such as IPtables, firewall, TCP wrappers, NMAP.
Environment: RHEL 5.x,6.x, Red-Hat Linux Enterprise servers (HP Proliant DL 585, BL 465/485,ML Series), Solaris 8,9,10, AIX 5.3, 6.1, SAN(Netapp), Blade Logic, VERITAS Cluster Server 5.0, Windows 2003 server, Shell programming, Jboss 4.2, JDK 1.5,1.6,, VMware Virtual Client 3.5, VMware Infrastructure 3.5.
Confidential, CA
Linux/Unix Administrator
Responsibilities:
- Installation, configuration and upgrade ofRedhat Linux3/4/5/6, Oracle Linux 5/6 Cent OS 5.7& operating systems.
- Involved in health check of servers after patching / rebooting and checked all the services of
- Extensively involved in troubleshooting bottlenecks of Memory, CPU, Devices and Network using ps, vmstat, iostat, netstat, sar, snoop, nfsstat and sar commands, tuning the performance.
- Responsible for the implementation of Authentication services for Linux likeCentrifyandWinbindto Authenticate to AD servers.
- Responsible for automating the process of setting up the Linux clients for AD authentication.
- Associated withDBAs, OBIEE, IAM, BRM, BI, OAMfor installation of Oracle database on Linux servers & for tuning, backup and restore.
- Scheduling the jobs by creating crontabs /Crontabs on Linuxand Windows Servers.
- Responsible for providing technical support and providing production support for internally developed application.
- Monitor the performance and availability of key middleware components in production and test environments.
- Have been the on-call person for the application, worked with the appropriate team members to bring the issue and get a resolution.
- Worked on customer tickets which are related to our application and help application team for the configuration changes and the file drops on the different servers.
- Troubleshooting and configuration of Local and Network based printers.User, Group Administrationand Advanced File Permissions.
- Maintain security for the networked environment while allowing users to access all data and software remotely.
- Involved in supportingApache, Tomcatand generated SSL keys for renewing SSL certs on web servers.
- Involved in developing custom scripts usingPython, Perl & Shell(bash, ksh) to automate jobs.
- Resolving production issues/tickets by using ticketing tool.
- Worked on installation and configuration and disk and file system management using Veritas Volume Manager(VxVM),Veritas Cluster Server(VCS).
- Performed Memory and Swap management to improve optimization and performance of the servers.
- Multipath configuration on SAN devices/disks on Linuxto keep the disks names persistent after reboot.
- Configuration of DNS servers on Linuxfor zones file, adding hosts to files.
- Performed upgrades of Packages and Patches in Oracle Linux and Linux.
- Installed and configuredOracle Real Application Cluster (RAC)on Linux Servers.
- Creation and scheduling cron jobs for backup, system monitoring and removal of unnecessary files.
- Worked on installation and configurationRedhat Cluster Suitein LinuxServers.
- Performed installation, configuration, upgrades, Package adminstrationand support for Linux systems on client side using Redhat satellite
- Installation and administration of RHEL 4.0/5.0 and SUSE 10.x.
- Configured kick start server and updating/applying patches to the servers using Red hat Satellite server.
- Remote system administration using tools like SSH, Telnet, and Rlogin.
- Planning and implementing system upgrades including hardware, operating system and periodical patch upgrades.
- Installation, configuration and administration of VMware, Solaris Zones, Containers and Ldoms.
- Automation of jobs through crontab and AutoSys.
- Installation of packages, patch management, volume management on Suse servers using YaST.
- Applied appropriate support packages/patches to maintain system integrity.
- Performed capacity analysis, monitored and controlled disk space usage on systems.
- Monitored system activities and fine-tuned system parameters and configurations to optimize performance and ensure security of systems.
- Adding servers to domain and managing the groups and user in AD, installing and configuring send mail.
- Custom build of Windows 2003 and Windows 2008 servers which includes adding users, SAN, network configuration, installing application related packages, managing services.
- Responsible for maintenance of development tools and utilities and to maintain shell, Perl automation Scripts.
- Worked with project manager and auditing teams to implement PCI compliance.
- Build LPAR and DLPAR system and Installed AIX 5.2, 5.3 on LPAR using NIM.
- Installed and configured Virtual I/O Server V1.3.01 with fixpack8.1.
- Integrating Web logic 10.x and Apache 2.x and deploying EAR, WAR files in Web logic Application servers.
- Management and monitoring of various services using Service Management Facility and Service Administration.
- Designing Firewall rules for new servers to enable communication with application, Oracle 10g servers.
- Working on Volume management, Disk Management, software RAID solutions using VERITAS Volume manager & Solaris Volume Manager. File system Tuning and growing using VERITAS File System (VxFS), coordinated with SAN Team for storage allocation and Disk Dynamic Multi path.
- Fine tuning of Servers and configuring networks for optimum performance.
- Organized various meetings with teams as mentioned above for tracking their requirements as inputs to get this done and designed the plan/steps involved and took user approvals.
- Setting up network environments using TCP/IP, NIS, NFS, DNS, SNMP agents, DHCP and Proxy.
- Installation, maintenance and administration of Oracle and Db2 Server on Sun Servers.
- Set up user and group login IDs, printing parameters, network configuration, password, resolving permissions issues, user and group quota.
- As a member of the team, monitored the VERITAS Cluster Server 4.1 in SAN Environment.
Environment: Solaris 8,9,10, Red Hat Linux AS/EL 4.0, AIX 5.2, 5.3, Sun E10k, E25K, E4500, SunFire V440/880, DMX 3 & DMX4, Site minder, Sonic MQ 7.0, VxFS 4.1, VxVM 4.1, SVM.
Confidential
UNIX Administrator
Responsibilities:
- Installation and configuring of SUN Solaris and Linux servers.
- Involved in Building and configuring Solaris 8/9/10 using Jump start server and Red Hat Linux Servers using Kick Start server as required for the project. Maintained maximum uptime and maximum performance capacity for Enterprise Production, QA, and UAT/Staging.
- Incident management for Unix Servers (Solaris, Linux).
- Installed and configured Solaris 10 servers with Zones and containers with Sun ZFS.
- Primary responsibility includes building and supporting Solaris/ Red hat Linux 3,4 servers.
- Monitoring CPU, memory, iSCSI disks, disk controllers, physical disk, HW and SW RAID, multipath, file systems, network using the tools NAGIOS and BMC Tools.
- Extensive experience in installing, integrating, tuning and troubleshooting Apache, Tomcat, Web Sphere and web logic application server.
- Implemented SUDO, SSH and other security tools and implemented SOX (Sarbanes Oxley) security compliance.
- Assisted in configuring the application servers with other components such as database, web servers, LDAP directory, Active Directory, BEA Web logic etc.
- Extensive use of VERITAS Volume Manager for Disk management, file system management on Sun Solaris environment.
- Created Zeta byte file system (ZFS) in Solaris 10. Created zpools. Exported ZFS from local zones to local zones.
- Installation and configuration VERITAS Netback up on Sun Severs and performing Backup using VERITAS Net Backup.
- Extensive use of Korn shell, Bash for scripting.
- Expertise in Linux backup/restore with tar including disk partitioning and formatting.
- Involved in Planning, building and Administration of various High Availability clusters and heart beat checking on Sun Solaris using VCS in heterogeneous SAN environment.
- Regular disk management like adding / replacing hot swappable drives on existing servers/workstations, partitioning according to requirements, creating new file systems or growing existing one over the hard drives and managing file systems.
- Planned, scheduled and Implemented OS patches on both Solaris & Linux boxes as a part of proactive maintenance.
- Worked with other IT teams, customers (users), and other managers in helping build and implement systems and standards.
- Involved in migration of projects from one flavor to another one.
- Involved in development, user acceptance, and performance testing, production & disaster recovery server
Environment: Solaris 8/9/10, Red Hat Linux 4/5, BMC Tools, NAGIOS, VERITAS Netback up, Korn Shell, Bash Scripting, VERITAS Volume Manager, web servers, LDAP directory, Active Directory, BEA Web logic servers, SAN Switches, Apache, Tomcat Web Sphere, web logic application server.
Confidential
Systems Administrator
Responsibilities:
- Installation, Configuration, up gradation and administration of Sun Solaris, Red Hat Linux.
- User account management and support.
- Jumpstart &Kick-start OS integration, DDNS, DHCP, SMTP, Samba, NFS, FTP, SSH, LDAP integration.
- Network traffic control, IPsec, Quos, VLAN, Proxy, Radius integration on Cisco Hardware via Red Hat Linux Software.
- Responsible for configuring and managing Squid server in Linux.
- Configuration and Administration of NIS environment.
- Managing file systems and disk management using Solstice Disk suite.
- Involved in Installing and configuring of NFS.
- Worked on Solaris volume manager to create file systems as per user and database requirements.
- Trouble shooting the system and end user issues.
- Responsible for configuring real time backup of web servers.
- Log file was managed for troubleshooting and probable errors.
- Responsible for reviewing all open tickets, resolve and close any existing tickets.
- Document solutions for any issues that have not been discovered previously
Environment: Sun Solaris 2.6/7, SUN Ultra Enterprise 6000/450, SUN Ultra 10/5/2/1, Windows NT 4.0, RHEL 3.x.