We provide IT Staff Augmentation Services!

Hadoop Admin Resume

NJ

SUMMARY:

  • Having 8 years of IT Experience in Analysis, Implementation and Testing of enterprise wide application, Data Warehouse, Client Server Technologies and Web - based Applications.
  • Over 6+ Years of experienced in administrative tasks such as Hadoop installation In pseudo distribution mode, multinode cluster and installation of Apache Ambari in Hortonworks Data Platform (HDP2.5).
  • Installation, configuration, supporting and managing Hortonworks Hadoop cluster.
  • Experience in working with cloud infrastructure like Amazon Web Services (AWS) and Rackspace.
  • Good Experience in setting up the Linux environments, Password less SSH, creating file systems, disabling firewalls and installing Java.
  • Experienced in design and implementations of robust technology systems, with specialized expertise in Hadoop, Linux and Network Administration.
  • Hadoop Cluster capacity planning, performance tuning, cluster Monitoring, Troubleshooting.
  • Design Big Data solutions for traditional enterprise businesses.
  • Involved in bench marking Hadoop/ Hbase cluster file systems various batch jobs and workloads.
  • Experience in minor and major upgrades of Hadoop and Hadoop eco system.
  • Experience in job scheduling using different schedulars like FAIR, CAPACITY & FIFO. and cluster co-ordination through DISTCP tool.
  • Administration of Hadoop and Vertica clusters for structured and unstructured data warehousing.
  • Administration of Hbase, Hive, Sqoop, HDFS, and MapReduce.
  • Hands on experience in analyzing Log files for Hadoop and eco system services and finding root cause.
  • Handsome experience in Linux admin activities on RHEL & Cent OS.
  • Experience in deploying Hadoop 2.0(YARN).
  • Monitor Hadoop cluster using tools like Nagios, Ganglia, Ambari and Cloudera Manager.
  • Familiar with writing Oozie workflows and Job Controllers for job automation.
  • Hands on experience in Zookeeper and ZKFC in managing and configuring in Name Node failure scenarios.
  • Experience in Amazon AWS cloud Administration and actively involved highly available, Scalability, cost effective and fault tolerant systems using multiple AWS services.
  • Experience in Migrating the On-Premise Data Center to AWS Cloud Infrastructure.
  • Ability to interact with developers and product analysts regarding issues raised and following up with them closely.
  • Experience in dealing with Hadoop cluster and integration with its Ecosystem like HIVE, HBase, PIG, SQOOP, Spark, OOZIE, Flume etc.
  • Experience in AWS CloudFront, including creating and managing distributions to provide access to S3 bucket or HTTP server running on EC2 instances.
  • Good working knowledge of Vertica DB architecture, column orientation, and High Availability.
  • Configured Informatica environment to connect to different databases using DB config, Input Table, Output Table, Update table Components.
  • Performed systems analysis for several information systems documenting and identifying performance and administrative bottlenecks.
  • Good understanding and extensive work experience on SQL and PL/SQL.
  • Experience in designing and implementing of secure Hadoop cluster using Kerberos.
  • Knowledge of all phases of Software Development Life Cycle ( SDLC ).

TECHNICAL SKILLS:

Operating Systems: UNIX/Linux (Redhat 3/4/5/6, Ubuntu), Windows Vista/XP/07/10

Database: Mysql, SQL,PL/SQL

Cassandra RDBMS: Oracle 9i, Oracle 10g, MS Access, MS SQL Server, IBM DB2.

Big Data Technologies: HDFS, Hive, Map Reduce, Pig, Sqoop, Oozie, Zookeeper, YARN, Avro, Spark

BI Reporting Tools: Tableau, Crystal Reporting and Power Pivot

Tools: Quality center v11.0\ALM,HP QTP,HP UFT, Selenium, Test NG, JUnit

QA methodologies: Waterfall, Agile, V-model.

Front End Technologies: HTML, XHTML, CSS, XML, JavaScript, AJAX, Servlets, JSP

Operating Systems: Linux, UNIX, MAC, Windows NT / 98 /2000/ XP / Vista, Windows 7,Windows 8.

Data Modeling: Star-Schema Modeling, Snowflakes Modeling, Erwin 4.0, Visio

RDBMS: Oracle 13.0, Teradata V2R6, Teradata 4.6.2, DB2, MS SQL Server 2000Programming: UNIX Shell Scripting, Korn Shell, SQL*Plus, PL/SQL,HTML

WORK EXPERIENCE:

Confidential, NJ

Hadoop Admin

Responsibilities:

  • Worked on installing cluster, commissioning & decommissioning of Data Nodes, Name Node recovery, capacity planning, Cassandra and slots configuration.
  • Installed and managed Hadoop production cluster with 350+ nodes with storage capacity of 10PB with HDP distribution using 1.7 Ambari and 2.1.3 HDP
  • Upgraded Production cluster from Ambari1.7 to 2.1 and HDP 2.1 to 2.2.6.
  • Experience in AWS Cloud Front, including creating and managing distributions to provide access to S3 bucket or HTTP server running on EC2 instances.
  • Configured custom metrics for the AWS Cloud Watch for detailed monitoring.
  • Maintained the architecture Hadoop 30 nodes Innovation Cluster with SQRRL, SPARK, Puppet, HDP 2.2.4.
  • Installing, Upgrading and Managing Hadoop Cluster on Hortonworks.
  • Setup, configured, and managed security for the Cloudera Hadoop cluster.
  • Hands on experience in installing, configuring Cloudera, MapR, Hortonworks clusters and installed Hadoop ecosystem components like Hadoop Pig, Hive, HBase, Sqoop, Kafka, Oozie, Flume, Zookeeper
  • Provided security and authentication with ranger where ranger admin provides administration and user sync adds the new users to the cluster.
  • Setting up MySQL master and slave replications and helping business applications to maintain their data in MySQL Servers.
  • Managed 350+ Nodes HDP 2.2.4 cluster with 4 petabytes of data using Ambari 2.0 and Linux Cent OS 6.5. Loaded the data from vertica to Hive using Sqoop.
  • Maintained and administrated HDFS through
  • Created Hive tables to store the processed results in a tabular format. Created 25+ Linux Bash scripts for users, groups, data distribution, capacity planning, and system monitoring.

Environment: Hive, Pig, HBase Apache Nifi, Java, Sqoop, Python, Ambari 2.0, Cent OS, HBase, MongoDB, Cassandra, Ganglia and Cloudera Manager.

Confidential

Hadoop Admin

Responsibilities:

  • Analyzed Hadoop cluster and other big data analysis tools using ganglia.
  • Experience in working with cloud infrastructure like Amazon Web Services (AWS) and Rackspace.
  • Implemented multiple nodes on CDH3 Hadoop cluster on Red hat Linux.
  • Developed Shell and Python scripts to automate and provide Control flow to Pig scripts. Imported data from Linux file system to HDFS
  • Worked on evaluating, architecting, installation/setup of Hortonworks 2.1/1.8 Big Data ecosystem which includes Hadoop, Pig, Hive, Sqoop etc.
  • Contributed to building hands-on tutorials for the community to learn how to setup Hortonworks Data Platform (powered by Hadoop) and Hortonworks Data flow (powered by NiFi).
  • Expertise in designing Python scripts to interact with middleware/back end services.
  • Designed a scalable Big Data clusters.
  • Experienced in Ambari-alerts configuration for various components and managing the alerts.
  • Provided security and authentication with ranger where ranger admin provides administration and user sync adds the new users to the cluster.
  • Good troubleshooting skills on Hue, which provides GUI for developer's/business users for day to day activities.
  • Extensively used mapping parameters and variables, post-Sql, pre-Sql, Sql overrides, lookup overrides in Informatica objects.
  • Used mapping Parameters and Variables for parameterizing the connections in workflow manager.
  • Tuned the performance of Informatica objects to load faster.
  • UNIX scripts to handle data quality issues and also to invoke the Informatica workflows.
  • Experience on creating and performance tuning of Vertica, Hive scripts.

Environment: HDFS CDH3, CDH4, Hbase, Python, RHEL 4/5/6, Hive, Pig, AWS S3, EC2, ganglia, Hadoop, HDFS, Pig, Sqoop, Hbase and Red Hat linux.

Confidential,Atlanta, GA

Hadoop Admin

Responsibilities:

  • Installed, configured and Administrated of all UNIX/LINUX servers, includes the design and selection of relevant hardware to Support the installation/upgradation of Red Hat and CentOS operating systems.
  • Network traffic control, IPsec, Quos, VLAN, Proxy, Radius integration on Cisco Hardware via Red Hat Linux Software.
  • Monitoring the Sqoop scripts in order to make the interaction between Hive and vertica Database
  • Written JCL, PARM, PROC for new processes and written Teradata BTEQ in JCL.
  • Used Agile/scrum Environment and used Jenkins, GitHub for Continuous Integration and Deployment. Projects also have other application integration to BI-DARTT.
  • Provisioning, building and support of Linux servers both Physical and Virtual using VMware for Production, QA and Developers environment.
  • Troubleshooting, Manage and review data backups, Manage and review Hadoop log files.
  • Deployed Datalake cluster with Hortonworks Ambari on AWS using EC2 and S3.
  • Hands on experience in installing, configuring Cloudera, MapR, Hortonworks clusters and installing Hadoop ecosystem components like Hadoop Pig, Hive, HBase, Sqoop, Kafka, Oozie, Flume and Zookeeper.
  • Worked on installing cluster, commissioning & decommissioning of Data Nodes, Name Node recovery, capacity planning, Cassandra and slots configuration.
  • Expertise with Hortonworks Hadoop platform(HDFS, Hive, Oozie, Sqoop, Yarn)
  • Installed and administered a Hadoop clusters consisting of 100 nodes.
  • Monitored cluster for performance and, networking and data integrity issues.
  • Responsible for troubleshooting issues in the execution of MapReduce jobs by inspecting and reviewing log files.
  • Performing Linux systems administration on production and development servers (Red Hat Linux, CentOS and other UNIX utilities)
  • Installation, Upgradation and administration of Sun Solaris, Red hat Linux.

Environment: Hadoop, Cassandra, Cloudera Manager, HDFS, Hive, Pig, HBase, Sqoop, Oozie, AWS, SQL, Java (JDK 1.6), Eclipse.

Confidential, Dallas, TX

Hadoop Admin

Responsibilities:

  • Experienced as admin in Hortonworks (HDP 2.2.4.2) distribution for clusters ranges from POC to PROD.
  • Good troubleshooting skills on Hue, which provides GUI for business users for day to day activities.
  • Migrated On Premise Data Center to AWS Cloud Infrastructure.
  • Supported AWS Cloud environment with 200+ AWS instances and configured Elastic IP & Elastic Storage and also experience working on implemented security groups.
  • Deploy, monitor, and maintain AWS cloud infrastructure consisting of multiple EC2 nodes as required in the environment.
  • Monitoring systems and services through Ambari dashboard to make the clusters available for the business. Worked with Nifi for managing the flow of data from source to HDFS.
  • Experienced in managing and reviewing Hadoop log files.
  • Provided security and authentication with ranger where ranger admin provides administration and user sync adds the new users to the cluster.
  • Setting up MySQL master and slave replications and helping business applications to maintain their data in MySQL Servers.
  • Experienced in Ambari-alerts configuration for various components and managing the alerts.
  • Involved in upgrading Hadoop Cluster from HDP 1.3 to HDP 2.0. Used AWS remote computing services such as S3, EC2.
  • Involved in loading data from UNIX file system to HDFS.
  • Shared responsibility for administration of Hadoop, Hive and Pig. Experience in DW concepts and technologies using Vertica application.
  • Administration of Hadoop and Vertica clusters for structured and unstructured data warehousing.

Environment: HDFS, Hive, Sqoop, Zookeeper and HBase,, Linux, Python, HDFS, Pig, Hive, HBase, Flume, Kafka, Sqoop, Shell Scripting.

Confidential, Coppell, TX

Hadoop Admin

Responsibilities:

  • Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage & review log files.
  • Experienced on adding/installation of new components and removal of them through Ambari.
  • Monitored multiple clusters environments using AMBARI Alerts, Metrics and Nagios.
  • Working experience on maintaining MySQL databases creation and setting up the users and maintain the backup of cluster metadata databases with corn jobs.
  • Setting up MySQL master and slave replications and helping business applications to maintain their data in MySQL Servers.
  • Engineer in Big Data team, worked with Hadoop, and its Ecosystem.
  • Having knowledge on Installation and configuration of Cloudera hadoop on production and development environment.
  • Experienced as admin in Hortonworks (HDP 2.2.4.2) distribution for clusters ranges from POC to PROD.
  • Implementing the SFTP for the projects to transfer data from External servers to servers.
  • Monitoring systems and services through Ambari dashboard to make the clusters available for the business.
  • Worked on setting up of environment and re-configuration activities. Participates in meetings with Client to understand business requirements in detail. Analyze and implement ETL frameworks and concepts Hadoop Admin.
  • This project involves File transmission and electronic data interchange, trades capture, verify, process and routing operations, Banking Reports Generation, Operational management.
  • Modified Oracle Packages, Procedures, functions, Triggers as per the business requirements.
  • DBMS include building data migration scripts using Oracle SQL LOADER.
  • Wrote and modified stored procedures to load and modifying of data according to business rule changes.
  • Worked on production support environment.

Environment: Apache Hadoop, Pig, Hive, SQOOP, Flume, Python, Java/J2EE, Oracle 11G, Jboss 5.1.0, Application Server, Linux OS, Windows OS.

Confidential

Linux Admin

Responsibilities:

  • Worked as System administration, maintenance and monitoring various day-to-day operations.
  • Well-Trained and worked Primarily on RHEL 5.x Operating Systems.
  • Experienced in Installation of Linux operating systems, applying Read, Write, and Execute file permission and on File system issues and Disk management.
  • Worked through Creating, Managing and modifying the user accounts, groups and access levels on Linux. Worked on package management using RPM and YUM.
  • Provided technical support by troubleshooting issues with various Servers on different platforms.
  • Notify server owner if there was a failover or crash. Also notify Unix Linux Server Support L3.
  • Monitored CPU loads, restart processes, check for file systems.
  • Installing, Upgrading and applying patches for UNIX, Red Hat/ Linux, and Windows Servers in a clustered and non-clustered environment.
  • Worked on Planning, configuring storage using LVM and applying patches on Linux machines.
  • Experienced on creating volume groups and Logical volumes on Linux.
  • Worked on Installation and Configuration of SAMBA server, DNS server APACHE server.
  • Worked on using tar command for Data Compressing, Backup and recovery.
  • Experienced in developing scripts in PERL and SHELL to automate the process, like Preparation of operational testing scripts for Log check, Backup and recovery and Failover.
  • Monitored server and application performance and tuned I/O, memory and Installation of SSH and configuring of keys base authentication.

Environment: Linux, Red Hat 5.x, DNS, YUM, RPM, LVM, PERL, SHELL, Samba, Apache, Tomcat Web-Sphere.

Confidential

Linux Admin

Responsibilities:

  • Provided 24x7 on-call supports in debugging and fixing issues related to Linux, Solaris, HP-UX Installation/Maintenance of Hardware/Software in Production, Development & Test Environment as an integral part of the Unix/Linux (RHEL/SUSE/SOLARIS/HP-UX/AIX) Support team.
  • Installation Red hat Linux Enterprise Server 5/6 on Dell and HP x86 HW.
  • Planning and implementing Backup and Restore procedures using Ufsdump, Ufsrestore, Tar" and "Cpio".
  • Installed and configured the Red Hat Linux 5.1 on HP-Dl585 servers using Kick Start.
  • Monitoring day-to-day administration and maintenance operations of the company network and systems working on Linux and Solaris Systems.
  • Configured the NIS, NIS+ and DNS on Red Hat Linux 5.1 and update NIS maps and Organize the RHN Satellite Servers in combination with RHN Proxy Server
  • Installed and configured the RPM packages using the YUM Software manager.
  • Involved in developing custom scripts using Shell (bash, ksh) to automate jobs.
  • Defining and Develop plan for Change, Problem & Incident management Process based on ITIL.
  • Networking communication skills and protocols such as TCP/IP, Telnet, FTP, NDM, SSH, rlogin.
  • Deploying Veritas Clusters and Oracle test databases to implement disaster recovery strategies, ensuring uninterrupted availability of the global systems.
  • Configured Logical storage Manager (LSM) for root disk encapsulation and implemented root disk mirroring for fault tolerance.
  • NIS, NFS, AUTOFS, NTP Configuration and Administration.
  • Define Mapping Servlets, Filters, and Session Configuration on JBOSS.
  • Worked on the administration of the Jboss servers including installation and deployments.
  • Also coordinating with storage team and networking teams.

Environment: GIT, GITHUB, Shell scripts, Maven, Jenkins, Ansible, Confluence, Jira, Virtual box, Vagrant LINUX (RHEL7, CentOS), UNIX, Server Spec.

Hire Now