We provide IT Staff Augmentation Services!

Senior Hadoop Administrator Resume

3.00/5 (Submit Your Rating)

Austin, TX

SUMMARY

  • Extensive experience and knowledge of processing BigData using Hadoop ecosystem components HDFS, Map Reduce, Hive, Pig, Sqoop, Oozie, Flume, NoSQL databases.
  • Very good experience in complete project life cycle (design, development, testing and implementation) of Client Server and Web applications.
  • Very Good experience on Unix, Linux, QSHELL scripting.
  • Well versed in installing, upgrading& managing Apache, Cloudera (CDH4) distributions for Hadoop.
  • Experience in Hadoop Distributions like Cloudera, HortonWorks, BigInsights, MapR Windows Azure, and Impala. Hands - on experience with Hadoop applications (such as administration, configuration management, monitoring, debugging, and performance tuning).
  • Strong knowledge on Hadoop HDFS architecture and MapReduce framework.
  • Responsible for writing MapReduce programs using Java.
  • Experience in troubleshooting errors in Pig, Hive and MapReduce.
  • Performed data analysis using Hive and Pig.
  • Experience in importing and exporting data between HDFS and Relational Database Management systems using Sqoop.
  • Good experience working with Hortonworks Distribution and Cloudera Distribution.
  • Implemented standards and processes for Hadoop based application design and implementation.
  • Strong understanding of NoSQL databases like HBase, MongoDB & Cassandra.
  • Developed MapReduce jobs to automate transfer of data from HBase.
  • Loaded streaming log data from various webservers into HDFS using Flume.
  • Support development, testing, and operations teams during new system deployments.
  • Experience in performing administrative tasks, Database files management, Backup/Restore, Security management, Data integration and Synchronization and Isolation levels.
  • Evaluate and propose new tools and technologies to meet the needs of the organization.

TECHNICAL SKILLS

  • HDFS, MapReduce, Sqoop,
  • Hive, PIG, HBASE, Flume, Oozie.
  • Hbase, Cassandra, mongoDB.
  • Java, C++, C, SQL, PIG Latin,
  • MATLAB, PL/SQL, MVC,
  • Maven, HTML, XML, TCP/IP,
  • HTTP, HTTPS, LINUX,
  • UNIX, UBUNTU,MacOS,
  • WINDOWS 98/00/NT/XP.
  • Oracle8i/9i, MY SQL,
  • MS SQL server

PROFESSIONAL EXPERIENCE

Senior Hadoop Administrator

Confidential, Austin, TX

Responsibilities:

  • Installing and configuring fully distributed Hadoop Cluster.
  • Installing Hadoop Eco-system Components (Pig, Hive and Hbase).
  • Involved in Hadoop Cluster environment administration that includes cluster capacity planning, performance tuning, cluster Monitoring and Troubleshooting.
  • Creating and configuring Hadoop cluster in Cloudera.
  • Coordinating and managing relations with vendors, IT developers and end users.
  • Managing the work streams, process and coordinate the team members and their activities to ensure that the technology solutions are in line with the overall vision and goals.
  • Design, implement and review features and enhancements to Cassandra.
  • Integrated Cassandra Querying Language called CQL for Apache Cassandra.
  • Installed and configured Hadoop MapReduce, HDFS, developed multiple MapReduce jobs in java for data cleaning and preprocessing.
  • Importing and exporting data into HDFS and Hive using Sqoop.
  • Configuration of various database connectivity (Oracle11g, SQL Server 2005).
  • Experienced in running Hadoop streaming jobs to process terabytes of xml format data.
  • Load and transform large sets of structured, semi structured and unstructured data.
  • Responsible to manage data coming from different sources.
  • Supported Map Reduce Programs those are running on the cluster.
  • Experience in providing security for Hadoop Cluster with Kerberos.
  • Cluster coordination services through Zoo Keeper.
  • Involved in loading data from UNIX file system to HDFS.
  • Installed and configured Hive and also written Hive UDFs.
  • Involved in creating Hive tables, loading with data and writing hive queries, which will run internally in map, reduce way.
  • Automated all the jobs, for pulling data from FTP server to load data into Hive tables, Using Oozie workflows.
  • Maintained, audited and built new clusters for testing purposes using the cloudera manager.

Environment:Cassandra, MapReduce, HDFS, Hive, Flume, Cloudera Manager, Sqoop MySQL, UNIX Shell Scripting, Zookeeper.

Senior Hadoop Administrator

Confidential, San Francisco,CA

Responsibilities:

  • Developed Simple to complex MapReduce Jobs.
  • Maintain the data in Accumulo using Java APIs and using SQRRL shell.
  • Create and publish REST Clients for the middleware to interact with the Accumulo NoSQL DB.
  • Create ETL jobs using the Talend Bigdata Studio.
  • Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from Postgres into HDFS using Sqoop.
  • Analyzed the data by performing Hive queries to study customer behavior.
  • Used UDF's to implement business logic in Hadoop.
  • Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
  • Installed and configured Hive, Pig, Sqoop and Oozie on the Hadoop cluster.
  • Installed Oozie workflow engine to run multiple MapReduce, Hive and Pig jobs.
  • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Monitor System health and logs and respond accordingly to any warning or failure conditions.
  • Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts.
  • Interact and co-ordinate with external teams to resolve technical issues.
  • Co-ordinate with clients to ensure delivery on time.

Environment: Hadoop, Map Reduce, HDFS, Hive, Postgres 9.6, Accumulo (SQRRL), Oozie, Java (jdk1.7), UNIX, SVN and Zookeeper.

Hadoop Administrator

Confidential, Woburn, MA

Responsibilities:

  • Capturing data from existing databases that provide SQL interfaces using Sqoop.
  • Implemented Hadoop stack and different bigdata analytic tools, migration from different databases to Hadoop.
  • Processed information from Hadoop HDFS. This information will comprise of various useful insights that can be used in the decision making process. All these insights will be presented to the users in the form of Charts.
  • Working on different Big Data technologies, good knowledge of Hadoop, Map-Reduce, Hive.
  • Developed various POCs over Hadoop, Big data.
  • Worked on deployments and automation task.
  • Installed and configured Hadoop cluster in pseudo and fully distributed mode environments.
  • Involved in developing the data loading and extraction processes for big data analysis.
  • Worked on professional services engagements to help customers design, build clusters, applications, troubleshoot network, disk and operating system related issues.
  • Administer linux servers, other UNIX variants, and managed hadoop clusters.
  • Installed and configured local Hadoop Cluster with 3 nodes and set up 4 nodes cluster on EC2 cloud.
  • Written MapReduce code to process and parsing the data from various sources and storing parsed data into HBase and Hive using HBase-Hive Integration.
  • Work with HBase and Hive scripts to extract, transform and load the data into HBase and Hive.
  • Continuous monitoring and managing of the Hadoop cluster.
  • Analyzed the data by performing Hive queries and running Pig scripts to know user behavior.
  • Installed Oozie workflow engine to run multiple Hive and Pig jobs.
  • Developing scripts and batch job to schedule a bundle (group of coordinators), which consists of various Hadoop programs using Oozie.
  • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports.

Environment: Hadoop, HDFS, Map Reduce, Hive, Flume, Sqoop, Cloudera CDH4, HBase, Oozie, Pig, AWS EC2 cloud.

Hadoop Administrator

Confidential, Framingham, MA

Responsibilities:

  • Involved in writing Map reduce programs and tested using MRUnit.
  • Managing and scheduling Jobs on a Hadoop cluster using Oozie.
  • Involved in moving all log files generated from various sources to HDFS for further processing through Flume.
  • Involved in loading data from UNIX file system to HDFS.
  • Worked on Hue interface for querying the data.
  • Created Hive tables to store the processed results in a tabular format.
  • Created HBase tables to store variable data formats of data coming from different portfolios.
  • Involved in transforming data from Mainframe tables to HDFS, and HBASE tables using Sqoop and Pentaho Kettle.
  • Implemented best income logic using Pig scripts.
  • Implemented test scripts to support test driven development and continuous integration.
  • Responsible to manage data coming from different sources.
  • Experienced on loading and transforming of large sets of structured, semi structured and unstructured data.
  • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
  • Participate in requirement gathering and analysis phase of the project in documenting the business requirements by conducting workshops/meetings with various business users.
  • Have deep and thorough understanding of ETL tools and how they can be applied in a Big Data environment.

Environment: Hadoop, HDFS, Pig, Sqoop, HBase, Shell Scripting, Ubuntu, Linux Red Hat.

Java Developer

Confidential

Responsibilities:

  • Involved in Requirement Analysis, Development and Documentation.
  • Participation in developing form-beans and action mappings required for struts implementation and validation framework using struts.
  • Development of front-end screens with JSP Using Eclipse.
  • XML and XSDs are used to define data formats.
  • Implemented J2EE design patterns value object singleton, DAO for the presentation tier, business tier and Integration Tier layers of the project.
  • Involved in Bug fixing and functionality enhancements.
  • Designed and developed excellent Logging Mechanism for each order process using Log4J.
  • Involved in writing Oracle SQL Queries.
  • Involved in Check-in and Checkout process using CVS.
  • Developed additional functionality in the software as per business requirements.
  • Involved in requirement analysis and complete development of client side code.

Environment:Java, J2EE, JSP, EJB, ANT, Weblogic 7.0, JDBC, MyEclipse, Windows XP, CVS, Oracle.

Java Developer

Confidential

Responsibilities:

  • End to end responsibility of Server management and deployment of packages
  • Involved in requirements gathering and analysis for VCare application
  • Involved in the project from its initiation and driven it through all the phases of SDLC like design, development and Testing (UT, ST, SIT and UAT)
  • Designing the workflow
  • Involved in the Performance Tuning of Tomcat Application Server for VCare application.
  • Implementation of Billing and Report Generation for VCare Application
  • Train the associates in PIM2R/VIBE framework
  • Created UML class diagrams that depict the code's design and its compliance with the functional requirements.
  • Used J2EE design patterns for the middle tier development.
  • Developed EJB's in Web Logic for handling business process, database access and asynchronous messaging.
  • Used Java Mail notification mechanism to send confirmation email to customers about scheduled payments.
  • Wrote stored procedures and Triggers using PL/SQL.
  • Involved in building and parsing XML documents using JAX parser.
  • Experience in implementing Web Services and XML/HTTP technologies.
  • Created Unix shell and Perl utilities for testing, data parsing and manipulation.

Environment:Java, JDBC, Servlets, JSP, Struts, Eclipse, Oracle 9i, CVS, JavaScript, J2EE, JDK6, Web Services, SQL, Log4j, XML, XSD, HTML, JUnit, CSS.

We'd love your feedback!