We provide IT Staff Augmentation Services!

Hadoop Engineer Resume

3.00/5 (Submit Your Rating)

Sunnyvale, CA

SUMMARY

  • Over 6+ years of experience in software development, 2+ years of experience in all phases of Hadoop and HDFS development.
  • Experience in developing Map Reduce Programs using Apache Hadoop for analyzing the big data as per the requirement.
  • Experienced on major Hadoop ecosystem’s projects such as PIG, HIVE and HBASE.
  • Good working experience using Sqoop to import data into HDFS from RDBMS and vice - versa
  • Good knowledge in using job scheduling and monitoring tools like Oozie and ZooKeeper
  • Experience in Hadoop administration activities such as installation and configuration of clusters using Apache and Cloudera
  • Experience in developing solutions to analyze large data sets efficiently
  • Knowledge of administrative tasks such as installing Hadoop and its ecosystem components such as Hive and Pig
  • Knowledge of NoSQL databases such as HBase, and MongoDB
  • In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS, JobTracker, TaskTracker, NameNode, Data Node and MapReduce concepts
  • Extending Hive and Pig core functionality by writing customUDFs
  • Good understanding of Data Mining and Machine Learning techniques
  • Experience in analyzing data using HiveQL, Pig Latin, and custom MapReduce programs in Java.
  • Strong work ethic with desire to succeed and make significant contributions to the organization
  • Strong problem solving skills, good communication, interpersonal skills and a good team player
  • Have the motivation to take independent responsibility as well as ability to contribute and be a productive team member

TECHNICAL SKILLS

Hadoop/Big Data Technologies: HDFS, MapReduce, Hive, Pig, Sqoop, Flume, Hbase, Oozie, Zookeeper, Kerberos

Programming Languages: Java JDK1.4/1.5/1.6 (JDK 5/JDK 6), C/C++, HTML, SQL, PL/SQL, AVS & JVS

Frameworks: Hibernate 2.x/3.x, Spring 2.x/3.x,Struts 1.x/2.x

Web Services: WSDL, SOAP, Apache CXF/XFire, Apache Axis, REST, Jersey

Client Technologies: JQUERY, Java Script, AJAX, CSS, HTML 5, XHTML

Operating Systems: UNIX, Windows, LINUX

Application Servers: IBM Web sphere, Tomcat, Web Logic, Web Sphere

Web technologies: JSP, Servlets, JNDI, JDBC, Java Beans, JavaScript, Web Services(JAX-WS)

Databases: Oracle 8i/9i/10g & MySQL 4.x/5.x

Java IDE: Eclipse 3.x, IBM Web Sphere Application Developer, IBM RAD 7.0

Tools: TOAD, SQL Developer, SOAP UI, ANT, Maven, Visio, Rational Rose, Endur 8.x/10.x/11.x

PROFESSIONAL EXPERIENCE

Confidential, Sunnyvale CA

Hadoop Engineer

Environment: Hadoop, MapReduce, HDFS, Hive, Pig, Java, SQL, Cloudera Manager, Sqoop, Flume, Oozie, Java (jdk 1.6), Eclipse

Responsibilities:

  • Responsible for building scalable distributed data solutions using Hadoop
  • Installed and configured Hive, Pig, Sqoop, Flume and Oozie on the Hadoop cluster
  • Setup and benchmarked Hadoop/HBase clusters for internal use
  • Developed Simple to complex Map/reduce Jobs using Hive and Pig
  • Worked with Python and Python web framework
  • Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms
  • Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop
  • Analyzed the data by performing Hive queries and running Pig scripts to study customer behavior
  • Used UDF’s to implement business logic in Hadoop
  • Implemented business logic by writing UDFs in Java and used various UDFs from Piggybanks and other sources.
  • Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from UNIX, NoSQL and a variety of portfolios.
  • Continuous monitoring and managing the Hadoop cluster using Cloudera Manager
  • Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required
  • Installed Oozie workflow engine to run multiple Hive and Pig jobs
  • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team

Confidential, Dallas, TX

Hadoop Developer

Environment: Hadoop, MapReduce, HDFS, Hive, Pig, Hue, Oozie, Core Java, Eclipse, Hbase, Flume, Cloudera Manager, Oracle 10g, DB2, IDMS, VSAM, SQL*PLUS, Toad, Putty, Windows NT, UNIX Shell Scripting, Pentaho Kettle, Pentaho Bigdata, YARN

Responsibilities:

  • Involved in writing Mapreduce programs and tested using MRUnit.
  • Managing and scheduling Jobs on a Hadoop cluster using Oozie.
  • Involved in moving all log files generated from various sources to HDFS for further processing through Flume.
  • Involved in loading data from UNIX file system to HDFS.
  • Worked on Hue interface for querying the data.
  • Created Hive tables to store the processed results in a tabular format.
  • Created HBase tables to store variable data formats of data coming from different portfolios.
  • Involved in transforming data from Mainframe tables to HDFS, and HBASE tables using Sqoop and Pentaho Kettle.
  • Implemented best income logic using Pig scripts.
  • Implemented test scripts to support test driven development and continuous integration.
  • Responsible to manage data coming from different sources.
  • Experienced on loading and transforming of large sets of structured, semi structured and unstructured data.
  • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
  • Participate in requirement gathering and analysis phase of the project in documenting the business requirements by conducting workshops/meetings with various business users.
  • Have deep and thorough understanding of ETL tools and how they can be applied in a Big Data environment.

Confidential

Java/J2EE Developer

Environment: Java, JDK 1.5, Servlets, Hibernate, Ajax, Oracle 10g, Eclipse, Apache Ant, Web Services (SOAP), Apache Axis, Apache Ant, Web Logic Server, JavaScript, HTML, CSS, XML

Responsibilities:

  • Responsible for gathering and analyzing requirements and converting them into technical specifications
  • Used Rational Rose for creating sequence and class diagrams
  • Developed presentation layer using JSP, Java, HTML and JavaScript
  • Used Spring Core Annotations for Dependency Injection
  • Designed and developed a ‘Convention Based Coding’ utilizing Hibernate’s persistence framework and O-R mapping capability to enable dynamic fetching and displaying of various table data with JSF tag libraries
  • Designed and developed Hibernate configuration and session-per-request design pattern for making database connectivity and accessing the session for database transactions respectively. Used HQL and SQL for fetching and storing data in databases
  • Participated in the design and development of database schema and Entity-Relationship diagrams of the backend Oracle database tables for the application
  • Implemented web services with Apache Axis
  • Designed and Developed Stored Procedures, Triggers in Oracle to cater the needs for the entire application. Developed complex SQL queries for extracting data from the database
  • Designed and built SOAP web service interfaces implemented in Java
  • Used Apache Ant for the build process
  • Used ClearCase for version control and ClearQuest for bug tracking

Confidential

Java/ J2EE Developer

Environment: Java, J2EE, XML, XML Schemas, JSP, HTML, CSS, PL/SQL, Junit, Log4j, IBM Web sphere Application Server.

Responsibilities:

  • Involved in creation of UML diagrams like Class, Activity, and Sequence Diagrams using modeling tools of IBM Rational Rose
  • Involved in the development of JSPs and Servlets for different User Interfaces
  • Used Struts action forms and developed Action Classes, which act as the navigation controller in Struts framework
  • Implemented the template-based categorization of presentation content using Struts-Tiles. MVC implementation using Struts framework
  • Involved in Unit Testing of Various Modules based on the Test Cases
  • Involved in Bug fixing of various modules that were raised by the Testing teams in the application during the Integration testing phase
  • Involved and participated in Code reviews
  • Used Log4J logging framework for logging messages
  • Used Rational ClearCase for version control
  • Used Rational Clear Quest for bug tracking
  • Involved in deployment of application on IBM Websphere Application Server

We'd love your feedback!