We provide IT Staff Augmentation Services!

Hadoop/cloudera Admin Resume

Allen, TX

SUMMARY:

  • Having 9 years of IT Experience in Analysis, Implementation and Testing of enterprise wide application, Data Warehouse, Client Server Technologies and Web - based Applications.
  • Over 5+ Years of experienced in administrative tasks such as Hadoop installation in pseudo distribution mode, multi node cluster.
  • Experience in deploying Hadoop 2.0 (YARN).
  • Administration of Hbase, Hive, Sqoop, HDFS, and MapR.
  • Installation of Apache Ambari in Hortonworks Data Platform (HDP2.5).
  • Installation, configuration, supporting and managing Hortonworks Hadoop cluster.
  • Experience in working with cloud infrastructure like Amazon Web Services and Rackspace.
  • Experience in understanding the security requirements for Hadoop and integrate with Kerberos authentication and authorization infrastructure.
  • Good knowledge on Kerberos Security while Successfully Maintained the cluster by adding and removal of nodes. Handsome experience in Linux admin activities on RHEL & Cent OS.
  • Experience in minor and major upgrades of Hadoop and Hadoop eco system.
  • Monitor Hadoop cluster using tools like Nagios, Ganglia, Ambari and Cloudera Manager.
  • Hadoop Cluster capacity planning, performance tuning, cluster Monitoring, Troubleshooting.
  • Involved in bench marking Hadoop/ Hbase cluster file systems various batch jobs and workloads.
  • Good Experience in setting up the Linux environments, Password less SSH, creating file systems, disabling firewalls and installing Java.
  • Experienced in design and implementations of robust technology systems, with specialized expertise in Hadoop, Linux and Network Administration.
  • Setting up MySQL master and slave replications and helping business applications to maintain their data in MySQL Servers.
  • Experience in job scheduling using different schedulers like FAIR, CAPACITY & FIFO and cluster co-ordination through DISTCP tool.
  • Administration of Hadoop and Vertica clusters for structured and unstructured data warehousing.
  • Hands on experience in analyzing Log files for Hadoop and eco system services and finding root cause.
  • Hands on experience in Zookeeper and ZKFC in managing and configuring in Name Node failure scenarios.
  • Experience in Amazon AWS cloud Administration and actively involved highly available, Scalability, cost effective and fault tolerant systems using multiple AWS services.
  • Ability to interact with developers and product analysts regarding issues raised and following up with them closely.
  • This project involves File transmission and electronic data interchange, trades capture, verify, process and routing operations, Banking Reports Generation, Operational management.
  • Experience in dealing with Hadoop cluster and integration with its Ecosystem like HIVE, HBase , PIG, SQOOP, Spark, OOZIE, Flume etc.
  • Experience in AWS CloudFront, including creating and managing distributions to provide access to S3 bucket or HTTP server running on EC2 instances.
  • Good working knowledge of Vertica DB architecture, column orientation and High Availability.
  • Configured Informatica environment to connect to different databases using DB config, Input Table, Output Table, Update table Components.
  • Performed systems analysis for several information systems documenting and identifying performance and administrative bottlenecks.
  • DBMS include building data migration scripts using Oracle SQL LOADER.
  • Good understanding and extensive work experience on SQL and PL/SQL
  • Experience in designing and implementing of secure Hadoop cluster using Kerberos.
  • Monitor health of the platforms and Generate Performance Reports and Monitor and provide continuous improvements.

TECHNICAL SKILLS:

Operating Systems: UNIX/Linux (Redhat 3/4/5/6, Ubuntu), Windows Vista/XP/07/10

Database: MySQL, SQL,PL/SQL

Cassandra RDBMS: Oracle 9i, Oracle 10g, MS Access, MS SQL Server, and IBM DB2.

Big Data Technologies: HDFS, Hive, Map Reduce, Pig, Sqoop, Oozie, Zookeeper, YARN, Avro, Spark

BI Reporting Tools: Tableau, Crystal Reporting and Power Pivot

Tools: Quality center v11.0\ALM, HP QTP, HP UFT, Selenium, Test NG, JUnit

QA methodologies: Waterfall, Agile, V-model.

Front End Technologies: HTML, XHTML, CSS, XML, JavaScript, AJAX, Servlets, JSP

Operating Systems: Linux, UNIX, MAC, Windows NT / 98 /2000/ XP / Vista, Windows 7, Windows 8.

Tools: AND Techniques:

Data Modeling Star: Schema Modeling, Snowflakes Modeling, Erwin 4.0, Visio

RDBMS: Oracle 13.0, Teradata V2R6, Teradata 4.6.2, DB2, MS SQL Server 2000

Programming: UNIX Shell Scripting, Korn Shell, SQL*Plus, PL/SQL,HTML

WORK EXPERIENCE:

Hadoop/Cloudera Admin

Confidential, Allen, TX

Responsibilities:

  • Primary tasks and responsibilities center on O&M support of a Secure (Kerberized) Cloudera distribution of Hadoop systems.
  • Installing and Configuring Systems for use with Cloudera distribution of Hadoop (consideration given to other variants of Hadoop such as Apache, MapR, Hortonworks, Pivotal, etc.)
  • Administering and Maintaining Cloudera Hadoop Clusters Provision physical Linux systems, patch, and maintain them.
  • Working with cloud infrastructure like Amazon Web Services (AWS) and Rackspace.
  • UNIX scripts to handle data quality issues and also to invoke the Informatica workflows.
  • Primarily using Cloudera Manager but some command-line. Designed a scalable Big Data clusters.
  • Experience in understanding the security requirements for Hadoop and integrating with Kerberos authentication infrastructure- KDC server setup, managing.
  • Integrated Hadoop with Active Directory and enabled Kerberos for Authentication.
  • Secured the Hadoop cluster from unauthorized access by Kerberos, LDAP integration and TLS for data transfer among the cluster node.
  • Involved in implementing security on HDF and HDF Hadoop Clusters with Kerberos for authentication and Ranger for authorization and LDAP integration for Ambar, Ranger, Nifi, Atlas, Grafana, KNOX and Zeppelin.
  • Experience in understanding the security requirements for Hadoop and integrating with Kerberos authentication infrastructure- KDC server setup, managing. Management and support of Hadoop Services including HDFS, Hive, Impala, and SPARK.
  • Person will be responsible to Perform Hadoop Administration on Production Hadoop clusters.
  • Monitored cluster for performance and, networking and data integrity issues.
  • Experience in working with cloud infrastructure like Amazon Web Services (AWS) and Rackspace.
  • Ensure the Hadoop platform can effectively meet performance & SLA requirements
  • Responsible for support of Hadoop Production environment which includes Hive, YARN, Spark, Impala, Kafka, SOLR, Oozie, Sentry, Encryption, Hbase, etc.
  • Configured Informatica environment to connect to different databases using DB config, Input Table, Output Table, Update table Components.
  • Perform optimization, capacity planning of a large multi-tenant cluster.

Environment: Cloudera 7.7, Hive, HDFS, Impala, and SPARK YARN, Kafka, SOLR, Oozie, Sentry, Encryption, Hbase, etc.

Hadoop Admin

Confidential, Bordentown, NJ

Responsibilities:

  • Worked on installing cluster, commissioning & decommissioning of Data Nodes, Name Node recovery, capacity planning, Cassandra and slots configuration.
  • Installed and managed Hadoop production cluster with 350+ nodes with storage capacity of 10PB with HDP distribution using 1.7 Ambari and 2.1.3 HDP
  • Upgraded Production cluster from Ambari1.7 to 2.1 and HDP 2.1 to 2.2.6.
  • Experience in AWS Cloud Front, including creating and managing distributions to provide access to S3 bucket or HTTP server running on EC2 instances.
  • Configured custom metrics for the AWS Cloud Watch for detailed monitoring.
  • Maintained the architecture Hadoop 30 nodes Innovation Cluster with SQRRL, SPARK, Puppet and HDP 2.2.4.
  • Installing, Upgrading and Managing Hadoop Cluster on Hortonworks.
  • Setup, configured, and managed security for the Cloudera Hadoop cluster.
  • Installing and configuring Kerberos for the authentication of users and Hadoop daemons.
  • Created system security supporting multi-tier software delivery system by utilizing Active Directory and Kerberos.
  • Experience in setup, configuration and management of security for Hadoop clusters using Kerberos and integration with LDAP/AD at an Enterprise level.
  • Managed 350+ Nodes HDP 2.2.4 cluster with 4 petabytes of data using Ambari 2.0 and Linux Cent OS 6.5. Loaded the data from Vertica to Hive using Sqoop.
  • Maintained and administrated HDFS through Created Hive tables to store the processed results in a tabular format. Created 25+ Linux Bash scripts for users, groups, data distribution, capacity planning, and system monitoring.

Environment: Hive, Pig, HBase Apache Nifi, Java, Sqoop, Python, Ambari 2.0, Cent OS, HBase, MongoDB, Cassandra, Ganglia and Cloudera Manager.

Hadoop Admin

Confidential, NYC

Responsibilities:

  • Analyzed Hadoop cluster and other big data analysis tools using ganglia.
  • Experience in working with cloud infrastructure like Amazon Web Services (AWS) and Rackspace.
  • Implemented multiple nodes on CDH3 Hadoop cluster on Red hat Linux.
  • Developed Shell and Python scripts to automate and provide Control flow to Pig scripts. Imported data from Linux file system to HDFS.
  • Integrated Hadoop with Active Directory and enabled Kerberos for Authentication.
  • Secured the Hadoop cluster from unauthorized access by Kerberos, LDAP integration and TLS for data transfer among the cluster node.
  • Integrated Hadoop cluster with Kerberos authentication infrastructure - KDC server set up, creating realm/domain, managing principals and generating key tab files for each service.
  • Worked on evaluating, architecting, installation/setup of Hortonworks 2.1/1.8 Big Data ecosystem which includes Hadoop, Pig, Hive, Sqoop etc.
  • Contributed to building hands-on tutorials for the community to learn how to setup Hortonworks Data Platform (powered by Hadoop) and Hortonworks Data flow (powered by Nifi).
  • Expertise in designing Python scripts to interact with middleware/back end services.
  • Experienced in Ambari-alerts configuration for various components and managing the alerts.
  • Provided security and authentication with ranger where ranger admin provides administration and user sync adds the new users to the cluster.
  • Good troubleshooting skills on Hue, which provides GUI for developer's/business users for day to day activities.
  • Extensively used mapping parameters and variables, post-Sql, pre-Sql, Sql overrides, lookup overrides in Informatica objects.
  • Used mapping Parameters and Variables for parameterizing the connections in workflow manager.
  • Tuned the performance of Informatica objects to load faster.
  • UNIX scripts to handle data quality issues and also to invoke the Informatica workflows.
  • Experience on creating and performance tuning of Vertica, Hive scripts.

Environment: HDFS CDH3, CDH4, Hbase, Python, RHEL 4/5/6, Hive, Pig, AWS S3, EC2, ganglia, Hadoop, HDFS, Pig, Sqoop, Hbase and Red Hat Linux.

Hadoop Admin

Confidential, Atlanta, GA

Responsibilities:

  • Installed, configured and Administrated of all UNIX/LINUX servers, includes the design and selection of relevant hardware to Support the installation/Upgradation of Red Hat and CentOS operating systems.
  • Network traffic control, IPsec, Quos, VLAN, Proxy, Radius integration on Cisco Hardware via Red Hat Linux Software.
  • Monitoring the Sqoop scripts in order to make the interaction between Hive and Vertica Database
  • Written JCL, PARM, PROC for new processes and written Teradata BTEQ in JCL.
  • Used Agile/scrum Environment and used Jenkins, GitHub for Continuous Integration and Deployment. Projects also have other application integration to BI-DARTT.
  • Experienced in Ambari-alerts configuration for various components and managing the alerts.
  • Deployed Datalake cluster with Hortonworks Ambari on AWS using EC2 and S3.
  • Hands on experience in installing, configuring Cloudera, MapR, Hortonworks clusters and installing Hadoop ecosystem components like Hadoop Pig, Hive, HBase, Sqoop, Kafka, Oozie, Flume and Zookeeper.
  • Worked on installing cluster, commissioning & decommissioning of Data Nodes, Name Node recovery, capacity planning, Cassandra and slots configuration.
  • Expertise with Hortonworks Hadoop platform(HDFS, Hive, Oozie, Sqoop, Yarn)
  • Installed and administered a Hadoop clusters consisting of 100 nodes.
  • Responsible for troubleshooting issues in the execution of MapReduce jobs by inspecting and reviewing log files.
  • Involved in upgrading Hadoop Cluster from HDP 1.3 to HDP 2.0. Used AWS remote computing services such as S3, EC2.
  • Performing Linux systems administration on production and development servers (Red Hat Linux, CentOS and other UNIX utilities)
  • Installation, Upgradation and administration of Sun Solaris, Red hat Linux.

Environment: Hadoop, Cassandra, Cloudera Manager, HDFS, Hive, Pig, HBase, Sqoop, Oozie, AWS, SQL, Java (JDK 1.6), Eclipse.

Hadoop Admin

Confidential, Dallas, TX

Responsibilities:

  • Experienced as admin in Hortonworks (HDP 2.2.4.2) distribution for clusters ranges from POC to PROD.
  • Good troubleshooting skills on Hue, which provides GUI for business users for day to day activities.
  • Migrated On Premise Data Center to AWS Cloud Infrastructure.
  • Supported AWS Cloud environment with 200+ AWS instances and configured Elastic IP & Elastic Storage and also experience working on implemented security groups.
  • Deploy, monitor, and maintain AWS cloud infrastructure consisting of multiple EC2 nodes as required in the environment. Installing and configuring Kerberos for the authentication of users and Hadoop daemons.
  • Created system security supporting multi-tier software delivery system by utilizing Active Directory and Kerberos.
  • Experienced in managing and reviewing Hadoop log files.
  • Monitoring systems and services through Ambari dashboard to make the clusters available for the business. Worked with Nifi for managing the flow of data from source to HDFS.
  • Experienced in managing and reviewing Hadoop log files.
  • Provided security and authentication with ranger where ranger admin provides administration and user sync adds the new users to the cluster.
  • Involved in upgrading Hadoop Cluster from HDP 1.3 to HDP 2.0. Used AWS remote computing services such as S3, EC2. Involved in loading data from UNIX file system to HDFS.
  • Shared responsibility for administration of Hadoop, Hive and Pig. Experience in DW concepts and technologies using Vertica application.
  • Administration of Hadoop and Vertica clusters for structured and unstructured data warehousing.

Environment: HDFS, Hive, Sqoop, Zookeeper and HBase, Linux, Python, HDFS, Pig, Hive, HBase, Flume, Kafka, Sqoop, Shell Scripting.

Hadoop Admin

Confidential, Coppell, TX

Responsibilities:

  • Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage & review log files.
  • Experienced on adding/installation of new components and removal of them through Ambari.
  • Monitored multiple clusters environments using AMBARI Alerts, Metrics and Nagios.
  • Working experience on maintaining MySQL databases creation and setting up the users and maintain the backup of cluster metadata databases with corn jobs.
  • Setting up MySQL master and slave replications and helping business applications to maintain their data in MySQL Servers.
  • Involved in implementing security on HDF and HDF Hadoop Clusters with Kerberos for authentication and Ranger for authorization and LDAP integration for Ambar, Ranger, Nifi, Atlas, Grafana, KNOX and Zeppelin.
  • Monitoring systems and services through Ambari dashboard to make the clusters available for the business. Worked with Nifi for managing the flow of data from source to HDFS.
  • Engineer in Big Data team, worked with Hadoop, and its Ecosystem.
  • Having knowledge on Installation and configuration of Cloudera Hadoop on production and development environment.
  • Monitoring systems and services through Ambari dashboard to make the clusters available for the business. Worked with Nifi for managing the flow of data from source to HDFS.
  • Experienced as admin in Hortonworks (HDP 2.2.4.2) distribution for clusters ranges from POC to PROD.
  • Implementing the SFTP for the projects to transfer data from External servers to servers.
  • Monitoring systems and services through Ambari dashboard to make the clusters available for the business.
  • Worked on setting up of environment and re-configuration activities. Participates in meetings with Client to understand business requirements in detail. Analyze and implement ETL frameworks and concepts Hadoop Admin. DBMS include building data migration scripts using Oracle SQL LOADER.
  • This project involves File transmission and electronic data interchange, trades capture, verify, process and routing operations, Banking Reports Generation, Operational management.
  • Wrote and modified stored procedures to load and modifying of data according to business rule changes.
  • Worked on production support environment.

Environment: Apache Hadoop, Pig, Hive, SQOOP, Flume, Python, Java/J2EE, Oracle 11G, Jboss 5.1.0, Application Server, Linux OS, Windows OS.

Linux Admin

Confidential

Responsibilities:

  • Worked as System administration, maintenance and monitoring various day-to-day operations.
  • Experienced in Installation of Linux operating systems, applying Read, Write, and Execute file permission and on File system issues and Disk management. Monitored CPU loads, restart processes, check for file systems.
  • Worked through Creating, Managing and modifying the user accounts, groups and access levels on Linux. Worked on package management using RPM and YUM.
  • Notify server owner if there was a failover or crash. Also notify Unix Linux Server Support L3.
  • Installing, Upgrading and applying patches for UNIX, Red Hat/ Linux, and Windows Servers in a clustered and non-clustered environment. Well-Trained and worked Primarily on RHEL 5.x Operating Systems.
  • Worked on Planning, configuring storage using LVM and applying patches on Linux machines.
  • Experienced on creating volume groups and Logical volumes on Linux.
  • Worked on Installation and Configuration of SAMBA server, DNS server APACHE server.
  • Worked on using tar command for Data Compressing, Backup and recovery.
  • Experienced in developing scripts in PERL and SHELL to automate the process, like Preparation of operational testing scripts for Log check, Backup and recovery and Failover.
  • Monitored server and application performance and tuned I/O, memory and Installation of SSH and configuring of keys base authentication.

Environment: Linux, Red Hat 5.x, DNS, YUM, RPM, LVM, PERL, SHELL, Samba, Apache, Tomcat Web-Sphere.

Linux Admin

Confidential

Responsibilities:

  • Provided 24x7 on-call supports in debugging and fixing issues related to Linux, Solaris, HP-UX Installation/Maintenance of Hardware/Software in Production, Development & Test Environment as an integral part of the Unix/Linux (RHEL/SUSE/SOLARIS/HP-UX/AIX) Support team.
  • Installation Red hat Linux Enterprise Server 5/6 on Dell and HP x86 HW.
  • Planning and implementing Backup and Restore procedures using Ufsdump, Ufsrestore, Tar" and "Cpio".
  • Installed and configured the Red Hat Linux 5.1 on HP-Dl585 servers using Kick Start.
  • Monitoring day-to-day administration and maintenance operations of the company network and systems working on Linux and Solaris Systems.
  • Configured the NIS, NIS+ and DNS on Red Hat Linux 5.1 and update NIS maps and organize the RHN Satellite Servers in combination with RHN Proxy Server.
  • OpenLdap server & clients, PAM authentication setup on RedHat Linux 6.5/7.1.
  • Installed, configured, troubleshoot and maintain Linux Servers and Apache Web server, configuration and maintenance of security and scheduling backups, submitting various types of croon jobs.
  • Installations of HP Open view, monitoring tool, in servers and worked with monitoring tools such as Nagios and HP Open view.
  • Installed and configured the RPM packages using the YUM Software manager.
  • Involved in developing custom scripts using Shell (bash, ksh) to automate jobs.
  • Defining and Develop plan for Change, Problem & Incident management Process based on ITIL.
  • Networking communication skills and protocols such as TCP/IP, Telnet, FTP, NDM, SSH, rlogin.
  • Deploying Veritas Clusters and Oracle test databases to implement disaster recovery strategies, ensuring uninterrupted availability of the global systems.
  • Also coordinating with storage team and networking teams.

Environment: GIT, GITHUB, Shell scripts, Maven, Jenkins, Ansible, Confluence, Jira, Virtual box, Vagrant LINUX (RHEL7, CentOS), UNIX, Server Spec.

Hire Now