We provide IT Staff Augmentation Services!

Hadoop Engineer Resume

4.00/5 (Submit Your Rating)

CT

SUMMARY

  • Over 7 years of strong IT experience in analysis, design, development and implementation of large - scale software applications. Inclusive of 3 years of hands-on experience in Big Data Technologies and Hadoop Stack along with 4 years of hands-on experience in development and design on Java and related frameworks
  • Around 3 years of experience as a Hadoop Developer with good knowledge of Hadoop framework, Hadoop Distributed file system and Parallel processing implementation.
  • Excellent understanding / knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programming paradigm.
  • Knowledge of the YARN architecture and its daemons such as Resource manager, Node Manager, Application master and containers.
  • Experience in Hadoop Ecosystems HDFS, Map Reduce, Hive, Pig, HBase, Sqoop and Flume.
  • Knowledge on Oozie workflow engine.
  • Knowledge of the Storm real time processing architecture.
  • Good Exposure on Apache Hadoop Map Reduce programming, Hive, PIG scripting and HDFS.
  • Experienced in importing and exporting data from relational database into HDFS using Scoop.
  • Strong experience in writing Map Reduce programs for Data Analysis.
  • Experience in writing UDFs for Hive and Pig.
  • Knowledge on creating new Key spaces and column families in Cassandra for transaction processing.
  • Experience in managing and reviewing Hadoop log files.
  • Experience in using Cloudera Manager for management of Hadoop Cluster.
  • Experience in Software Development Life Cycle (Requirements Analysis, Design, Development, Testing, Deployment and Support).
  • Hands on experience in application development using Java, RDBMS and Linux shell scripting.
  • Experience working with JAVA J2EE, JDBC, ODBC, JSP, Servlets, Hibernate, HTML, CSS, JavaScript and JQuery.
  • Experience in using IDEs like Eclipse and Visual Studio.
  • Strong Experience in working with Databases like Oracle 11g/10g/9i,DB2, SQL Server 2008 and MySQL and proficiency in writing complex SQL queries.
  • Knowledge of data warehousing, including ETL Processes, Cube, Rollup, Window and Ranking based queries.
  • Hands on experience on writing Queries, Stored procedures, Functions and Triggers by using SQL.
  • Support development, testing, and operations teams during new system deployments.
  • Evaluate and propose new tools and technologies to meet the needs of the organization.
  • An excellent team player and self-starter with good communication skills and proven abilities to finish tasks before target deadlines.

TECHNICAL SKILLS:

Big Data Ecosystem: HDFS, HBase, Hadoop MapReduce, Hive,Pig, Sqoop, Spark, Flume, Oozie, Cassandra, Storm, Impala.

Distributions: Apache Hadoop 1.0.4, Cloudera CDH3, CDH4.

Languages: C, C++, Java, SQL/PLSQL, Python.

Methodologies: Agile, waterfall.

Database: Oracle 10g, DB2, MySQL, MS SQL server.

Web Tools: HTML, Java Script, XML, ODBC, JDBC, Hibernate, JSP,Servlets, Java, Struts, Springs, Junit, Json, Avro.

IDE / Testing Tools: Eclipse, Visual Studio, NetBeans, Putty.

Operating System: Windows, UNIX, Linux.

Scripts: JavaScript, Shell Scripting.

Version Control: SVN, CVS, TFS.

PROFESSIONAL EXPERIENCE

Confidential, CT

Hadoop Engineer

Responsibilities:

  • Designed and implemented MapReduce based large-scale parallel processing
  • Handled importing of data from various data sources, performed transformations using Hive, MapReduce, and loaded data into HDFS
  • Experienced working with real time analytical operations using HBase
  • Implemented CRUD operations on HBase Data using HBase Client Java API and Rest API.
  • Integrated HBase with MapReduce to work on bulk data
  • Continuous monitoring and managing the Hadoop cluster through Cloudera Manager
  • To set up standards and processes for Hadoop based application design and implementation
  • Written Hive queries for data analysis to meet the business requirements
  • Creating Hive tables and working on them using Hive QL
  • Extending HIVE and PIG core functionality by using UDF's
  • Experienced with performance tuning against Hive operations
  • Experience in Designing, developing and implementing connectivity products that allow efficient exchange of data between our core database engine and Hadoop ecosystem
  • Experience in importing and exporting the data using Sqoop from HDFS to Relational Database systems/mainframe and vice-versa

Environment: Java, Eclipse, Linux, Apache Hadoop 1.0.3, MapReduce, HBase, Sqoop, PIG, Hive, Flume, Oracle 10g

Confidential, Chattanooga, TN

Hadoop Developer/Engineer

Responsibilities:

  • Installed and configured Hadoop and responsible for maintaining cluster. Managed and reviewed Hadoop log files.
  • Provided support for simple and complex MapReduceprograms in Java for Data Analysis.
  • Migrating the needed data from MySQL into HDFS using Sqoop and importing various formats of unstructured data from logs into HDFS using Flume.
  • Developed PIG scripts to modify and clean the data received.
  • Worked extensively onHive and PIG.
  • Worked on large sets of structured, semi-structured and unstructured data.
  • DevelopedPIG Latin scriptsto retrieve information from the data.
  • Involved in creatingHive tables, loading with data and writing hive queries which will run internally inmap reduce way.
  • Involved in writing custom HiveUDFs in Java where the functionality is complex.
  • Involved in writing SerDe regular expressions in order to read unstructured data from various sources into hive tables.
  • Responsible to manage and analyze data coming from different sources.
  • Used Avro file format to store query results in order to manage large quantity of data.

Environment:ApacheHadoop, HDFS, JavaMapReduce, Eclipse, Hive, PIG, Sqoop, Flume, Oozie and HQL

Confidential, Warren, NJ

Java/J2EE Developer

Responsibilities:

  • Developed all the User Interfaces usingJSPandSpring MVCframework.
  • Writing Client Side validations using JavaScript.
  • Extensively usedJQueryfor developing interactive web pages.
  • Developed the DAO layer usingthe hibernateand for real time performance used the caching system for hibernate.
  • Experience in developingwebservicesfor production systems using SOAP and WSDL.
  • Developed the user interface presentation screens using HTML, XML and CSS.
  • Experience in working withspringusingAOP,IOCandJDBCtemplate.
  • Developed the Shell scripts to trigger the JavaBatchjob, Sending summary email for the batch job status and processing summary.
  • Co-ordinate with the QA lead for development of test plan, test cases, test code and actual testing responsible for defects allocation and those defects are resolved.
  • Involved in performing maintenance on the existing code base developed in thespringandHibernateframework by incorporating new features and doing bug fixes.
  • Used ANT tool to build and deploy applications.
  • Involved in configuring web.xml for workflow.
  • Wrote SQL queries and created DDL scripts for interacting with the Oracle database.
  • Involved in writing Stored Procedures in Oracle to do some database side validations.
  • Was part of solving the issues that came inproduction.
  • Documentation of common problems prior to go-live and while actively in a Production Support role.
  • Participated in Hadoop training and development as a part of cross training program.

Environment:J2SE, J2EE, Eclipse 3.2, Spring 2.5, Hibernate 3.0, JSP, XML, JUnit, Web logic 10.3, JavaScript, Oracle 10g, HTML, AJAX, JQuery, CSS.

Confidential

Java Developer

Responsibilities:

  • Involved in designing of shares and cash modules using UML.
  • Effectively used the iterative waterfall model software development methodology during this time constraint project.
  • Used HTML and JSP for the web pages and used JavaScript for Client side validation.
  • Created XML pages with DTD’sfor front-end functionality and information exchange.
  • Responsible for writing Java SAX parsers programs.
  • Developed ANT build scripts to build and deploy application in enterprise archive format (.ear)
  • Performed Unit testing using JUnit and Functional Testing.
  • Used the Json response format to retrieve data from web servers.
  • Used JDBC 2.0 extensively and was involved in writing several SQL queries for the data retrieval.
  • Prepared program specifications for the loans module and involved in database designing.
  • Java programming using swing to complete the functionality of cash lockers and security modules.
  • Servlet programming for connecting to the database server and to retrieve the serialized data.
  • Programmed stored procedures using SQL and PL/SQL for the bulk calculations of general ledger.

Environment:Java, J2EE, EJB 2.0, Servlets, JavaScript, OO, JSP, JNDI, Java Beans, Web Logic, XML, XSL, Eclipse, PL/SQL, Oracle 8i, HTML, DHTML, UML.

Confidential

Java Developer

Responsibilities

  • Worked with Relational Database Management Systems including Oracle, SQL Server and PL/SQL and have worked extensively on JDBC to connect to and access various remote databases including Oracle, My Sql, SQL Server.
  • Used Spring AOP, Spring IOC frameworks extensively during the development.
  • Designed and implemented MVC2, DAO, DTO Design Patterns.
  • Implemented JAVA/J2EE design patterns such as Factory, DAO, Session Façade, and Singleton.
  • Implemented J2EE Design Patterns like MVC, Service Locator and Session Facade.
  • Developed Web services to communicate to other modules using XML based SOAP.
  • Developed Message Handler Adapter, which converts the data objects into XML message and invoke an enterprise service and vice-versa using Java, JMS and MQ Series.
  • Business logic is implemented using Struts action components in the Struts and Hibernate framework.
  • Used Multithreading for invoking the database and also implemented complex modules which contain business logics using Collection, Reflection, and Generics API.
  • Developed various JSP custom tag libraries i.e. JSTL libraries for achieving most code-reusability.
  • Involved in implementation of the presentation layer (GUI) for the application using HTML, XHTML, CSS and JavaScript.
  • Used Hibernate in persistence layer and developed POJO's, Data Access Object (DAO) to handle all database operations.
  • Wrote JUNIT test cases for testing all spring service calls and Spring MVC validations
  • Created stored procedures to extract data from various data sources like Flat Files, XML Files and SQL server.
  • Responsible for designing the application using the MVC architecture and Other Design patterns.
  • Responsible for designing user interface screens using HTML, EXTJS, JSP and validating using Java Script
  • Developed the Action classes, Action forms and Action Servlets for Web tier.
  • Used Java script in user validation and suggestion list and to display dialog boxes.
  • Working experience inCoreJava, Struts, XML, SQLSERVER, DB2, Mongo DB, REST web services, Spring, Hibernate, CXF
  • Worked on creating CSS style, JavaScript and AJAX.
  • Wrote spring services in the service layer for Oracle data retrieval operations using Hibernate.

Environment: Core Java, J2ee,XML, JSP, HTML,CSS, JavaScript, JUNIT, Oracle, Multithreading, PL/SQL, Spring 2.5, Struts 2.x, Hibernate, MVCEDUCATION

We'd love your feedback!