Hadoop Developer Resume
New York, NY
SUMMARY
- Around 8 years of professional IT experience in software systems Development client/server, Business systems and web based applications with over 2.5+ years of experience in the Big Data ecosystem related technologies.
- Hands - on-experience on major components in Hadoop Ecosystem Hive, Hbase, NoSql, Hbase-Hive Integration, PIG, Sqoop and knowledge of Mapper/Reducer/HDFS Framework and YARN.
- Experience in data management and implementation of Big Data applications using Hadoop frameworks.
- Experience in importing and exporting data using Sqoop for Hadoop to/from RDBMS.
- Very Good knowledge and Hands-on experience in Cassandra, Flume and Spark.
- Good knowledge in distributed coordination system ZooKeeper and experience with Data Warehousing and ETL.
- Exposure to Cloudera development environment and management using Cloudera Manager.
- Good knowledge on DataStax Enterprise, DataStax OpsCenter and CQL and Strong database, SQL, ETL and data analysis skills.
- Excellent understanding of Hadoop architecture and its components such as HDFS, JobTracker, TaskTracker, NameNode, DataNode and MapReduce programming paradigm.
- Experience in analyzing data using HiveQL, Pig Latin, Hbase and custom MapReduce programs in Java.
- Experience in extending Hive and Pig core functionality by writing custom UDFs using Java.
- Extensive experience in developing applications using JSP, Servlets, JavaBeans, JSTL, JSP Custom Tag Libraries, JDBC, EJB, MVC, Struts, Spring, Hibernate, JMS, Ant, Maven, HTML, Jquery, JavaScript, XML and XSLT.
- Experience in creating complex SQL Queries and SQL tuning, writing PL/SQL blocks like stored procedures, Functions, Cursors, Index, triggers and packages.
- Good experience in implementation and testing of Web Services using SOAP and REST based architecture.
- Expertise working on Application servers and IDEs like BEA WebLogic, IBM Websphere, JBOSS and Apache Tomcat, Eclipse, RAD
- Expertise in configuring, using & administrating version control systems like GIT, Clear Case, Subversion and CVS.
- Expert knowledge over J2EE Design Patterns like MVC Architecture, Front Controller, Session Facade, Business Delegate and Data Access Object for building J2EE Applications.
- Expertise in preparing the test cases, documenting and performing unit testing and Integration testing.
- Excellent technical, logical, code debugging and problem solving capabilities and ability to watch the future environment, the competitor and customers probable activities carefully.
- Good team player with strong analytical and communication skills.
TECHNICAL SKILLS
Big Data Ecosystem: Hadoop, MapReduce, HDFS, HBase, Hive, Pig, Sqoop, Flume, Oozie, Zookeeper, Spark
Operating Systems: Windows, Ubuntu, RedHat Linux
Programming/ Scripting Languages: Java, Python, Unixshell scripting, JavaScript, Ruby, Python, Perl
Databases/Database Languages: MySQL, Oracle 9i/11g, NoSQL (Hbase, Cassandra), SQL, PL/SQL
Web Technologies: JSP, Servlet, JSF, Ajax, Jquery, HTML, CSS, Java Beans, EJB, SOAP, REST, WSDL, XML, JSON
Web Servers: Tomcat, WebLogic, Jboss, WebSphere
ETL: Informatica, Teradata
Frameworks: J2EE, Hibernate, Spring, Apache Maven, Struts
IDEs: Eclipse, NetBeans, RAD
Testing tools: JUnit, Mockito, MRUnit
PROFESSIONAL EXPERIENCE
Confidential
Hadoop Developer
Responsibilities:
- Responsible for building scalable distributed data solutions using Hadoop.
- Involved in loading data from Oracle database into HDFS using Sqoop queries.
- Utilized Agile Scrum Methodology to help manage and organize a team of 5 developers and actively participated in scrum meetings.
- Created the Hbase Tables and inserted data into it.
- Documented the systems processes and procedures for future references.
- Create ETL Pipelines using Apache Crunch.
- Researched on Cassandra NoSQL database architecture for data read/write.
- Worked on connecting to a 5-node Cassandra cluster from java using DataStax Java Driver and retrieved the query results.
- Created tables, inserted data and executed various Cassandra Query Language (CQL 3) commands on tables from java code and using cqlsh command line client .
- Used DataStax OpsCenter, a web-based tool to monitor and to simplify administration tasks in Cassandra.
- Writing Mapreduce jobs to standardize the data and clean it and calculate aggregates.
- Implemented test scripts to support test driven development and continuous integration.
- Mainly worked on Hive queries to categorize data of different claims.
- Integrated the hive warehouse with Hbase.
- Written customized Hive UDFs in Java where the functionality is too complex.
- Creating Oozie workflows to manage the execution of the MR jobs
- Actively participated in software development lifecycle (scope, design, implement, deploy, test), including design and code reviews, test development, test automation.
- Used Zookeeper for providing coordinating services to the cluster.
- Created Hive based reports to support the application metrics which will be used by UI team for reports.
- Maintained System integrity of all sub-components (primarily HDFS, MR, HBase, and Hive).
- Monitored System health and logs and respond accordingly to any warning or failure conditions.
Environment: Apache Hadoop, MapReduce, HDFS, Hive, Java, Sql, Sqoop, Oracle, PL/SQL, Hbase, Crunch, Zookeeper, Cassandra, Oozie, CQL 3, DataStax Enterprise, DataStax OpsCenter, UNIX shell scripting, Eclipse.
Confidential, New York, NY
Hadoop Developer
Responsibilities:
- Installed and configured HadoopMapreduce, HDFS, developed multiple MapReduce jobs in java for data cleaning and preprocessing.
- Extracted the needed data from the server into HDFS and BulkLoaded the cleaned data into HBase using MapReduce.
- Migrating the needed data from MySQL in to HDFS using Sqoop and importing various formats of flat files in to HDFS.
- Weekly meetings with technical collaborators and active participation in code review sessions with senior and junior developers.
- Analyzed large data sets to determine optimal way to aggregate and report on it.
- Written the Map Reduce programs, Hive UDFs in Java.
- Provide batch processing solution to certain unstructured and large volume of data by using Hadoop Map Reduce framework.
- Developed Map Reduce programs to cleanse the data in HDFS obtained from heterogeneous data sources to make it suitable for ingestion into Hive schema for analysis.
- Implemented Partitioning, Dynamic Partitions, Buckets in HIVE for efficient data access.
- Used Map Reduce JUnit for unit testing.
- Develop HIVE queries for the analysts.
- Used Oozie Scheduler system to automate the pipeline workflow and orchestrate the map reduces jobs that extract the data on a timely manner.
- Created an e-mail notification service upon completion of job for the particular team which requested for the data.
- Moving data from Hive tables into HBase for real time analytics on hive tables.
- Worked with business teams and created Hive queries for ad hoc analysis.
- Used Flume and Sqoop to load data from multiple sources into HDFS to be processed by Pig and Hive to create a more usable data store.
- Played a key role in productionizing the application after testing by BI analysts.
Environment: Apache Hadoop, HDFS, Hive, Map Reduce, Java, Cloudera CDH4, Oozie, Oracle, MySQL, Pig, Hbase, Junit, Flume, Sqoop.
Confidential, Alpharetta, GA
Java/J2ee Developer
Responsibilities:
- Involved in writing JSP and JSF components. Used JSTL Tag library (Core, Logic, Nested, and Bean and Html taglib’s) to create standard dynamic web pages.
- Application was based on MVC architecture with JSP serving as presentation layer, Servlets as controller and Hibernate in business layer to access to Oracle Database.
- Developed the DAO layer for the application using Spring Hibernate Template support.
- Developing the business functionalities and exporting them as REST web services.
- Used XML and JSON for transferring/retrieving data between different Applications.
- Used the Spring Dependency Injection and developed the presentation layer using Struts MVC Framework.
- Used JQuery for creating JavaScript behaviors.
- Business-logic controller framework development utilizing spring framework and dependency injection, catering to both standard server-side and AJAX client requests.
- Worked on Apache Velocity, a Java-based Template Engine for merging of different templates.
- Collaborated with technical architects to ensure that the design meets the requirements.
- Implemented the Jboss server logging configuration which is represented by the logging subsystem.
- Implemented the Guava cache in the project for storing the template name as value for a particular key.
- Implemented EJB Session Beans and MDB.
- Used SVN for version control of the source code.
- Responsible for developing JUnit test cases using Easy Mock and DB units for unit and integration units.
- Used Maven script for building and deploying the application.
- Assisted in development and improvement of application maintenance plans, processes, procedures, standards and priorities.
Environment: Java, J2EE, HTML, Spring 3, Hibernate 3, Web Services, Jboss, Java SDK, SOA, JavaScript, JMS, Struts 2.0, REST, JSON, Servlet 2.2, JSF, JSP, CSS3, HTML, DHTML, JQuery, AJAX, JDBC, Oracle, Clear Case, ANT 1.5, TOAD and Log4J 1.2, UML, JAXB, XML, Scrum, Unix/Linux.
Confidential, Chicago, IL
Java Software Engineer
Responsibilities:
- Application was developed using the Struts MVC architecture.
- Developed action and form classes based on Struts framework to handle the pages.
- Developed a web-based reporting for credit monitoring system with HTML5, XHTML, JSTL, custom tags and Tiles using Struts framework.
- Understanding and analyzing business requirements, High Level Design and Detailed Design.
- Provided high level systems design; this includes specifying the class diagrams, sequence diagrams and activity diagrams
- Utilized Java/J2EE Design Patterns - MVC Confidential various levels of the application and ATG Frameworks
- Expertise in developing JSP’s, Servlets and good with web services (REST, SOAP)
- Served as DB Administrator, creating and maintaining all schemas
- Collaborated in design, development and maintenance of the Front-end for applications using JSP, JSTL, Custom Tags.
- Developed Servlets and JSPs based on MVC pattern using Struts framework.
- Developing Web Services using Apache Axis 2 to retrieve data from legacy systems.
- Developed Servlets, Action classes, Action Form classes and configured the struts-config.xml file.
- Used XML parser APIs such as JAXP and JAXB in the web service's request response data marshalling as well as unmarshalling process.
- Planned and implemented various SQL, Stored Procedure, and triggers.
Environment: J2EE, Java 1.5, Servlets, JSP, JDBC, JQuery, backbone.js, HTML5, JSTL, XML, Struts, Hibernate, Web Services, WebLogic Server, JSF, JAXB, Jasper Report, JUnit, SOAP, XML, JavaScript, UML, Apache Axis 2, ANT, MySQL.