Hadoop Engineer Resume
CT
SUMMARY
- Over 7 years of strong IT experience in analysis, design, development and implementation of large - scale software applications. Inclusive of 3 years of hands-on experience in Big Data Technologies and Hadoop Stack along with 4 years of hands-on experience in development and design on Java and related frameworks
- Around 3 years of experience as a Hadoop Developer with good knowledge of Hadoop framework, Hadoop Distributed file system and Parallel processing implementation.
- Excellent understanding / knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programming paradigm.
- Knowledge of the YARN architecture and its daemons such as Resource manager, Node Manager, Application master and containers.
- Experience in Hadoop Ecosystems HDFS, Map Reduce, Hive, Pig, HBase, Sqoop and Flume.
- Knowledge on Oozie workflow engine.
- Knowledge of the Storm real time processing architecture.
- Good Exposure on Apache Hadoop Map Reduce programming, Hive, PIG scripting and HDFS.
- Experienced in importing and exporting data from relational database into HDFS using Scoop.
- Strong experience in writing Map Reduce programs for Data Analysis.
- Experience in writing UDFs for Hive and Pig.
- Knowledge on creating new Key spaces and column families in Cassandra for transaction processing.
- Experience in managing and reviewing Hadoop log files.
- Experience in using Cloudera Manager for management of Hadoop Cluster.
- Experience in Software Development Life Cycle (Requirements Analysis, Design, Development, Testing, Deployment and Support).
- Hands on experience in application development using Java, RDBMS and Linux shell scripting.
- Experience working with JAVA J2EE, JDBC, ODBC, JSP, Servlets, Hibernate, HTML, CSS, JavaScript and JQuery.
- Experience in using IDEs like Eclipse and Visual Studio.
- Strong Experience in working with Databases like Oracle 11g/10g/9i,DB2, SQL Server 2008 and MySQL and proficiency in writing complex SQL queries.
- Knowledge of data warehousing, including ETL Processes, Cube, Rollup, Window and Ranking based queries.
- Hands on experience on writing Queries, Stored procedures, Functions and Triggers by using SQL.
- Support development, testing, and operations teams during new system deployments.
- Evaluate and propose new tools and technologies to meet the needs of the organization.
- An excellent team player and self-starter with good communication skills and proven abilities to finish tasks before target deadlines.
TECHNICAL SKILLS:
Big Data Ecosystem: HDFS, HBase, Hadoop MapReduce, Hive,Pig, Sqoop, Spark, Flume, Oozie, Cassandra, Storm, Impala.
Distributions: Apache Hadoop 1.0.4, Cloudera CDH3, CDH4.
Languages: C, C++, Java, SQL/PLSQL, Python.
Methodologies: Agile, waterfall.
Database: Oracle 10g, DB2, MySQL, MS SQL server.
Web Tools: HTML, Java Script, XML, ODBC, JDBC, Hibernate, JSP,Servlets, Java, Struts, Springs, Junit, Json, Avro.
IDE / Testing Tools: Eclipse, Visual Studio, NetBeans, Putty.
Operating System: Windows, UNIX, Linux.
Scripts: JavaScript, Shell Scripting.
Version Control: SVN, CVS, TFS.
PROFESSIONAL EXPERIENCE
Confidential, CT
Hadoop Engineer
Responsibilities:
- Designed and implemented MapReduce based large-scale parallel processing
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce, and loaded data into HDFS
- Experienced working with real time analytical operations using HBase
- Implemented CRUD operations on HBase Data using HBase Client Java API and Rest API.
- Integrated HBase with MapReduce to work on bulk data
- Continuous monitoring and managing the Hadoop cluster through Cloudera Manager
- To set up standards and processes for Hadoop based application design and implementation
- Written Hive queries for data analysis to meet the business requirements
- Creating Hive tables and working on them using Hive QL
- Extending HIVE and PIG core functionality by using UDF's
- Experienced with performance tuning against Hive operations
- Experience in Designing, developing and implementing connectivity products that allow efficient exchange of data between our core database engine and Hadoop ecosystem
- Experience in importing and exporting the data using Sqoop from HDFS to Relational Database systems/mainframe and vice-versa
Environment: Java, Eclipse, Linux, Apache Hadoop 1.0.3, MapReduce, HBase, Sqoop, PIG, Hive, Flume, Oracle 10g
Confidential, Chattanooga, TN
Hadoop Developer/Engineer
Responsibilities:
- Installed and configured Hadoop and responsible for maintaining cluster. Managed and reviewed Hadoop log files.
- Provided support for simple and complex MapReduceprograms in Java for Data Analysis.
- Migrating the needed data from MySQL into HDFS using Sqoop and importing various formats of unstructured data from logs into HDFS using Flume.
- Developed PIG scripts to modify and clean the data received.
- Worked extensively onHive and PIG.
- Worked on large sets of structured, semi-structured and unstructured data.
- DevelopedPIG Latin scriptsto retrieve information from the data.
- Involved in creatingHive tables, loading with data and writing hive queries which will run internally inmap reduce way.
- Involved in writing custom HiveUDFs in Java where the functionality is complex.
- Involved in writing SerDe regular expressions in order to read unstructured data from various sources into hive tables.
- Responsible to manage and analyze data coming from different sources.
- Used Avro file format to store query results in order to manage large quantity of data.
Environment:ApacheHadoop, HDFS, JavaMapReduce, Eclipse, Hive, PIG, Sqoop, Flume, Oozie and HQL
Confidential, Warren, NJ
Java/J2EE Developer
Responsibilities:
- Developed all the User Interfaces usingJSPandSpring MVCframework.
- Writing Client Side validations using JavaScript.
- Extensively usedJQueryfor developing interactive web pages.
- Developed the DAO layer usingthe hibernateand for real time performance used the caching system for hibernate.
- Experience in developingwebservicesfor production systems using SOAP and WSDL.
- Developed the user interface presentation screens using HTML, XML and CSS.
- Experience in working withspringusingAOP,IOCandJDBCtemplate.
- Developed the Shell scripts to trigger the JavaBatchjob, Sending summary email for the batch job status and processing summary.
- Co-ordinate with the QA lead for development of test plan, test cases, test code and actual testing responsible for defects allocation and those defects are resolved.
- Involved in performing maintenance on the existing code base developed in thespringandHibernateframework by incorporating new features and doing bug fixes.
- Used ANT tool to build and deploy applications.
- Involved in configuring web.xml for workflow.
- Wrote SQL queries and created DDL scripts for interacting with the Oracle database.
- Involved in writing Stored Procedures in Oracle to do some database side validations.
- Was part of solving the issues that came inproduction.
- Documentation of common problems prior to go-live and while actively in a Production Support role.
- Participated in Hadoop training and development as a part of cross training program.
Environment:J2SE, J2EE, Eclipse 3.2, Spring 2.5, Hibernate 3.0, JSP, XML, JUnit, Web logic 10.3, JavaScript, Oracle 10g, HTML, AJAX, JQuery, CSS.
Confidential
Java Developer
Responsibilities:
- Involved in designing of shares and cash modules using UML.
- Effectively used the iterative waterfall model software development methodology during this time constraint project.
- Used HTML and JSP for the web pages and used JavaScript for Client side validation.
- Created XML pages with DTD’sfor front-end functionality and information exchange.
- Responsible for writing Java SAX parsers programs.
- Developed ANT build scripts to build and deploy application in enterprise archive format (.ear)
- Performed Unit testing using JUnit and Functional Testing.
- Used the Json response format to retrieve data from web servers.
- Used JDBC 2.0 extensively and was involved in writing several SQL queries for the data retrieval.
- Prepared program specifications for the loans module and involved in database designing.
- Java programming using swing to complete the functionality of cash lockers and security modules.
- Servlet programming for connecting to the database server and to retrieve the serialized data.
- Programmed stored procedures using SQL and PL/SQL for the bulk calculations of general ledger.
Environment:Java, J2EE, EJB 2.0, Servlets, JavaScript, OO, JSP, JNDI, Java Beans, Web Logic, XML, XSL, Eclipse, PL/SQL, Oracle 8i, HTML, DHTML, UML.
Confidential
Java Developer
Responsibilities
- Worked with Relational Database Management Systems including Oracle, SQL Server and PL/SQL and have worked extensively on JDBC to connect to and access various remote databases including Oracle, My Sql, SQL Server.
- Used Spring AOP, Spring IOC frameworks extensively during the development.
- Designed and implemented MVC2, DAO, DTO Design Patterns.
- Implemented JAVA/J2EE design patterns such as Factory, DAO, Session Façade, and Singleton.
- Implemented J2EE Design Patterns like MVC, Service Locator and Session Facade.
- Developed Web services to communicate to other modules using XML based SOAP.
- Developed Message Handler Adapter, which converts the data objects into XML message and invoke an enterprise service and vice-versa using Java, JMS and MQ Series.
- Business logic is implemented using Struts action components in the Struts and Hibernate framework.
- Used Multithreading for invoking the database and also implemented complex modules which contain business logics using Collection, Reflection, and Generics API.
- Developed various JSP custom tag libraries i.e. JSTL libraries for achieving most code-reusability.
- Involved in implementation of the presentation layer (GUI) for the application using HTML, XHTML, CSS and JavaScript.
- Used Hibernate in persistence layer and developed POJO's, Data Access Object (DAO) to handle all database operations.
- Wrote JUNIT test cases for testing all spring service calls and Spring MVC validations
- Created stored procedures to extract data from various data sources like Flat Files, XML Files and SQL server.
- Responsible for designing the application using the MVC architecture and Other Design patterns.
- Responsible for designing user interface screens using HTML, EXTJS, JSP and validating using Java Script
- Developed the Action classes, Action forms and Action Servlets for Web tier.
- Used Java script in user validation and suggestion list and to display dialog boxes.
- Working experience inCoreJava, Struts, XML, SQLSERVER, DB2, Mongo DB, REST web services, Spring, Hibernate, CXF
- Worked on creating CSS style, JavaScript and AJAX.
- Wrote spring services in the service layer for Oracle data retrieval operations using Hibernate.
Environment: Core Java, J2ee,XML, JSP, HTML,CSS, JavaScript, JUNIT, Oracle, Multithreading, PL/SQL, Spring 2.5, Struts 2.x, Hibernate, MVCEDUCATION