We provide IT Staff Augmentation Services!

Sr Hadoop Administrator Resume

0/5 (Submit Your Rating)

San Francisco, CA

SUMMARY

  • Highly skilled Hadoop Administrator has extensive noledge of multiple scripting and programming languages.
  • Possesses strong abilities in administration of large data clusters in big data environments and is extremely analytical wif excellent problem - solving.
  • Possess transferable skills developed on large scale projects, across multiple industry sectors, predominantly Finance, Banking, Telecommunications, Healthcare and Sales.
  • Strong business acumen, thrives in diverse and challenging environments, as well as demonstrates a high-level of consistent achievement. Excellent client service skills grant teh ability to liaise and negotiate at all levels.
  • Having 7 years of experience in Project life cycle, design, development, testing and implementation of moderate to advanced complex systems.
  • 3+ years of experience in Hadoop Administration & Big Data Technologies and 4 years of experience into Linux administration.
  • Hands on experience in installation, configuration, supporting and managing Hadoop Clusters using Apache, Cloudera (CDH3, CDH4),Yarn distributions.
  • Primary technical skills inApache Hadoop, Map-Reduce, Pig, Hive, Hbase, Zookeeper, Sqoop, Flume, OOZIE, Core Java, Java Script, J2EE, Oracle 11G/10G, HP PPM 9.2.
  • Hands on experience in creating various database objects like tables, stored procedures, functions, and triggers using SQL, PL/SQL,DB2.
  • Strong exposure to IT consulting, software project management, team leadership, design, development, implementation, maintenance/support and Integration of Enterprise Software Applications.
  • Strong experience in System Administration, Installation, Upgrading, Patches, Migration, Configuration, Troubleshooting, Security, Backup, Disaster Recovery, Performance Monitoring and Fine-tuning on UNIX (SUN Solaris, Red Hat Linux)Systems.
  • Hands on experience in provisioning and managing multi-tenant Hadoop clusters on public cloud environment - Amazon Web Services (AWS)-EC2 and on private cloud infrastructure - Open Stack cloud Platform.
  • Experience wifWindows Active Directories, Exchange, DHCP and DNS configurations.
  • Extensive experience in SAP BI-BO/BPC/HANA/CRM/HR/ABAP/FICO/SD/MM-WM/SQL Data warehouse in all stages of Project Development from Blueprint Feasibility Analysis to Technical Design to Realization/Development to Go Live Support and business User Training.
  • Good experience on Design, configure and manage teh backup and disaster recovery for Hadoop data.
  • Experience in administering Tableau and Green Plum databases instances in various environments.
  • Hands on experience in analyzingLog files for Hadoop and eco system servicesand finding root cause.
  • Experience on Commissioning, Decommissioning, Balancing, and Managing Nodes and tuning server for optimal performance of teh cluster.
  • Extensive experience onHbase and zookeeper.
  • Experience in importing and exporting teh data usingSqoop from HDFSto Relational Database systems/mainframe and vice-versa.
  • Optimizing performance of Hbase/Hive/Pig jobs.
  • Experience in designing and implementing security for Hadoop cluster wif Kerberos secure autantication.
  • Hands on experience onNagios and Ganglia tool.
  • Experience in scheduling all Hadoop/hive/Sqoop/Hbase jobs using Oozie.
  • Experience in Databases such asOracle 11G/10G, DB2, Teradata, MySQLandSybase.
  • Having working noledge onOracle Administration.
  • Having working noledge onWindowsandUNIXshell scripting.
  • Working in different IDE's like EditPlus3, Eclipse3.5, IntelliJ IDEA 7.2andNetbeans6.5.
  • Knowledge ofData Ware Housingconcepts andCognos 8 BI SuitandBusiness Objects.
  • Experience in preparing teh impact analysis document for teh new enhancements and up- gradation process.
  • Experience in migration of objects across different environments using Deployment management module.
  • Working on development projects, which include design, development and unit testing of applications.
  • Working on production and support projects, which include reporting and prioritizing teh issues and defects and resolve them.
  • Understand and implement teh Software development life cycleSDLCprocess.
  • Team Player wif good communication and interpersonal skills and also goal oriented approach to problem solving issues.
  • Flexible on working on various technologies.

TECHNICAL SKILL

Big Data Ecosystems: Hadoop, MapReduce, HDFS, HBase, Zookeeper, Hive, Pig, Sqoop, Kerberos, Cassandra, Oozie, Flume, Pentaho, Kettle and Talend

Programming Languages: Java, C/C++, .NET

Scripting Languages: JSP & Servlets, PHP, JavaScript, XML, HTML, Python and Bash

Databases: NoSQL, MongoDB, Oracle, SQL, DB2

UNIX Tools: Apache, Yum, RPM

Tools: SAP BI 7.0, Eclipse, JDeveloper, JProbe, CVS, Ant, Tableau, MS Visual Studio

Platforms: Windows(2000/XP), Linux, Solaris, AIX, HPUX

Application Servers: Apache Tomcat 5.x 6.0, Jboss 4.0

Testing Tools: NetBeans, Eclipse, WSAD, RAD

Methodologies: Agile, UML, SDLC, Design Patterns

PROFESSIONAL EXPERIENCE

Confidential, San Francisco, CA

Sr Hadoop Administrator

Responsibilities:

  • As an admin involved in Cluster maintenance, trouble shooting, Monitoring and followed proper backup& Recovery strategies.
  • Installed, configured, secured, and troubleshoot Hortonworks Hadoop Data Platform (HDP).
  • Installation and configuration, Hadoop Cluster and Maintenance, Cluster Monitoring and Troubleshooting and Transform data from RDBMS to HDFS
  • Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and
  • Designed and implemented Hadoop clusters for various clients in both on premises physical or virtual platforms as well as cloud deployments.
  • Experience wif securing Hadoop clusters including Kerberos KDC installation, OpenLDAP installation, private x509 certificate authority creation, data transport encryption wif TLS, and data-at-rest encryption wif Cloudera Navigator Encrypt.
  • Extracted teh data from MySQL into HDFS using Sqoop Performed data completeness, correctness, data transformation and data quality testing using SQL
  • Exported teh analyzed data to teh relational databases using Sqoop for visualization and to generate reports for teh BI team.
  • Involved in various POC activity using technology like Map reduce, Hive, Pig, and Oozie.
  • Involved in designing and implementation of service layer over HBase database.
  • Developed various dashboards in Tableau, used context filters, sets while dealing wif huge volume of data.
  • Importing of data from various data sources such as Oracle and Comptel server into HDFS using transformations such as Sqoop, Map Reduce.
  • Analyzed teh data by performing Hive queries and running Pig scripts to no user behavior like frequency of calls, top calling customers.
  • Continuous monitoring and managing teh Hadoop cluster through Cloudera Manager.
  • Developed Hive queries to process teh data and generate teh data cubes for visualizing.
  • Designed and developed scalable and custom Hadoop solutions as per dynamic data needs and Coordinated wif technical team for production deployment of software applications for maintenance.
  • Created scripts to form EC2 clusters for training and for processing.
  • Implemented performance monitoring tools (HP)
  • Worked on Amazon cloud Migration project.
  • Worked for Amazon Elastic Cloud project using Agile Methodologies.
  • Assisted business analyst in posting migration project.
  • Reviewed firewall settings (security group) and updated on Amazon AWS.
  • Created access documents for level/tier 3/4 production support groups.
  • Created Cassandra Advanced Data Modeling course for DataStax.
  • Experience in providing support to data analyst in running Pig and Hive queries.
  • Managing and reviewing Hadoop log files.
  • Installing and configuring Hive and also written Hive UDFs
  • Experience in large scale data processing, on an Amazon EMR cluster
  • Efficient to handled Hadoop admin and user command for administration.
  • Supported technical team members for automation, installation and configuration tasks.
  • Wrote shell scripts to monitor teh health check of Hadoop daemon services and respond accordingly to any warning or failure conditions
  • Involved in data extraction and conversion between BPC and non-ERP systems wif extractors and ETL tools.
  • Developed custom Process chains to support master data and transaction data loads from BI to BPC.
  • Created Package links to bundle more TEMPthan one packages and to run sequentially.
  • Involved in creating Hive tables, loading wif data and writing hive queries which will run internally in map reduce way.

Environment: Hadoop, HDFS, Hive, Sqoop, Flume, Tableau, Zookeeper and HBase, Oracle 9i/10g/11g RAC wif Solaris/redhat, Exadata MachinesX2/X3, Kerberos, Big Data Cloudera CDH Apache Hadoop, Toad, SQL plus, Oracle Enterprise Manager (OEM), Shell Scripting, Golden Gate, Redhat/Suse Linix, EM Cloud Control

Confidential, Rapid City, SD

Hadoop Administrator

Responsibilities:

  • Identified teh best solutions/ Proof of Concept leveraging Big Data & Advanced Analytics levers dat meet and exceed teh customer's business, functional and technical requirements.
  • Strong working experience wif open source technology
  • Store unstructured data in semi structure on HDFS using Hbase
  • Loading log data directly into HDFS using Flume.
  • Experienced in managing and reviewing Hadoop log files.
  • Installation of various Hadoop Ecosystems and Hadoop Daemons.
  • Used Change management and Incident management process following teh company standards
  • Implemented partitioning, dynamic partitions and buckets in H IVE
  • Continuous monitoring and managing teh Hadoop cluster through Cloudera Manager
  • Demonstration of teh Live Proof Of Concept Demo to Clients
  • Supported technical team members in management and review of Hadoop log files and data backups.
  • Developing interactive graph visualization tool based on Prefuse vis package.
  • Developing machine-learning capability via Apache Mahout.
  • Working wif data delivery team to setup new Hadoop users, Linux users, setting up Kerberos TEMPprincipals and testing HDFS, Hive, Pig and MapReduce access for teh new users on Horton works & Cloudera Platform.
  • Research effort to tightly integrate Hadoop and HPC systems.
  • Deployed, and administered 70 node Hadoop cluster. Administered two smaller clusters.
  • Compared Hadoop to commercial big-data appliances from Netezza, XtremeData, and LexisNexis. Published and presented results.
  • Suggested improvement processes for all process automation scripts and tasks.

Environment: Apache Hadoop, Pig, Hive, Map-reduce, Sqoop, UNIX, LINUX, Oracle 11gR2, JAVA/J2EE, UNIX shell scripting, Kerberos, WINDOWS.

Confidential

Linux Administrator

Responsibilities:

  • Involved in design and ongoing operation of several Hadoop clusters.
  • Implemented and operated on-premises Hadoop clusters from teh hardware to teh application layer including compute and storage.
  • Configured and deployed hive metastore using MySQL and thrift server.
  • Designed custom deployment and configuration automation systems to allow for hands-off management of clusters via Cobbler, FUNC, and Puppet.
  • Fully automated teh configuration of firmware, teh deployment of teh operating system, and teh configuration of teh OS and applications resulting in a less TEMPthan twenty minute server deployment time.
  • Deployed teh company's first Hadoop cluster running Cloudera's CDH2 to a 44 node cluster storing 160TB and connecting via 1 GB Ethernet.
  • Collaborated wif application development teams to provide operational support, platform expansion, and upgrades for Hadoop Infrastructure including upgrades to CDH3.
  • Participated in Hadoop development Scrum.
  • Member of internal standards bodies addressing network and server strategic direction and architecture.
  • Responsible for maintaining Linux platform build standards including all aspects of OS configuration and deployment.
  • Submitted improvements to teh Solaris standard builds.
  • Wrote documentation and mentored other System Administrators.

Environment: Red Hat Linux 4, Sun Solaris 10/9, Intel servers, Perl and Shell Scripting, Sun Ultra Enterprise 4500, Sun Enterprise 450, Sun Enterprise 420R, Sun Enterprise 250, Sun Fire V20z servers, Dell PCs & Compaq PCs, VERITAS Volume Manager, SAP BI, VERITAS Netbackup 5.0, Sybase 11.5.1, Perl Scripting, Apache.

We'd love your feedback!