We provide IT Staff Augmentation Services!

Hadoop/spark Developer Resume

3.00/5 (Submit Your Rating)

PhiladelphiA

SUMMARY

  • Seasoned Java/Hadoop/Spark/Scala developer with 7+ years of experience in Object Oriented programming.
  • 5+ Years hands on experience on Java/J2EE technologies
  • 2.6+ years hands on experience on Hadoop (HDFS, Map Reduce, PIG, HIVE, and SQOOP).
  • 1+ year’s hands on experience on Spark (1.5, 1.6) & Scala Full stack developer.
  • Work experience on top different domains Banking, Financial, Health Care, Ecommerce and Media.
  • Worked on a live 480 nodes Hadoop cluster running Horton Works and CDH4.4
  • Good knowledge on Akka, Angular JS, Node JS, React JS, NoSQL (Mongo DB, Cassandra) and Kafka
  • Moving data from HDFS to RDBMS and vice - versa using SQOOP, Spark.
  • Good knowledge on HBase, OOZIE, Mongo DB, Cassandra, HUE, SPLUNK, YARN.
  • Experience in using Text, Parquet, ORC, Sequence files, AVRO file formats.
  • Proficient in the development of client/server and web Applications using JAVA, J2EE, J2SE technologies and Web Technologies for enterprise applications.
  • Expert in developing enterprise web-based application based on Java technologies using J2EE, J2SE, Java Servlets, Springs, Struts, Java Beans, JMS, JNDI, ANT and Apache Maven.
  • Expertise in developing web based GUIs using Java Applets, HTML, DHTML, JavaScript, XML, DOJO, AJAX, JSP, EXT-Js, Angular JS.
  • Good working knowledge on versioning tools CVS, GIT etc...
  • Hands on experience in configuration and deployment of multi-tier applications using servers like WebSphere, Web Logic and JBoss.
  • Extensive experience to SOA (Service Oriented Architecture) and experience in developing Web services REST and SOAP using Apache CXF, WSDL, SOAP, WS Security, Apache POI, XML-RPC and JAXB/JAXP.
  • Extensive experience working in Spring 2.5/3.0 framework, Spring batch, Struts 1.0/1.1/2.x framework, O/R Mapping Hibernate 3.x framework.
  • Strong experience in writing SQLs, Stored Procedures and Triggers.
  • Working experience in relational databases like ORACLE, Tera Data and SQL Server.
  • Good Working Knowledge in Ant, Maven, SBT for project build/test/deployment, Log4j for logging and JUnit for unit and integration testing.
  • Strong knowledge of IDEs like Eclipse, IBM Rational Application Developer (RAD 7.0.0) IntelliJ.
  • Extensive experience in software development Agile methodologies.
  • Excellent team player with leadership abilities. Strong analytical skills with excellent interpersonal and communication skills with analytical ability and understanding of Organizational functioning.

TECHNICAL SKILLS

Programming Technologies: JDK1.6/1.7/1.8, Scala, Akka, JSP 1.2/2, Servlets, Ajax, JSF, Hibernate3.0, Spring2.X, Spring3.X, JDBC, JUnit, JNDI, XML, JMS, Web Services, REST, SOAP,JSP,JSTL,XSLT

Big Data/Distributed Data Management: Spark, Hadoop, Map Reduce, YARN, HIVE, PIG, HBASE, SQOOP, HDP (Horton), CDH, Kafka

UI Technologies: HTML, Java Script, DOJO, Ajax,JSON, CSS, JQuery, Angular JS, Node JS, React

Web/App Servers: IBM Websphere 5.1/6.x, Bea Web Logic Server 9.2, Tomcat 4.0/5.0, Apache1.3/2.0, JBoss 4.x

RDBMS: Oracle 10g, 11g, Mysql, TeraData

Operating System: Windows XP/7/10, Linux, Unix

Development Tools: Eclipse, IntelliJ, RAD6.x, 7.x, Jet Brains Web Storm, IBM WebSphere 5.x/6.x, IBM MQ Series, TOAD, ANT, MAVEN, SQL Loader, IBM Messaging Queue series(MQ).

PROFESSIONAL EXPERIENCE

Confidential, Philadelphia

Hadoop/Spark Developer

Responsibilities:

  • Used Agile methodology in developing the application, weekly Sprints, stand up meetings.
  • Worked on a live 480 nodes Hadoop cluster running on HDP 2.2.4
  • Worked with highly unstructured and semi structured data of 2PB in size (TB of data with replication factor of 3)
  • Developed Spark Jobs pipeline which are used to stream, transformations, aggregations and generating the daily snapshots of customer data.
  • Developed UI component and REST web services using React JS and Scala
  • Developed Map Reduce Jobs (using Deprecated and MR2) to generate the Flat files on data from multiple sources like hive table, text files data.
  • Developed MR code in Eclipse IDE using JAVA, Multiple Inputs, Multiple Outputs, Aggregations, and Joins.
  • Extracted the RDBMS (Oracle, TERADATA etc..)Data into HDFS and Hive table using SQOOP and Shell scripts.
  • Created and worked SQOOPjobs with incremental load to populate Hive External tables.
  • Developed Hive(version 0.14) scripts and written HQL’s for ETL process on data.
  • Very good understanding of Partitions, Bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance
  • Worked on posting messages on Kafka (Producer and Consumer) topics.
  • Developed REST service to post the messages on Kafka topics.
  • Working experience with both MapReduce 1 (Job Tracker) and MR2 (YARN)
  • MR jobs and Work flow jobs monitoring and managing using HUE and UC4.
  • Developed Test cases, Test scenarios & Test scripts for Unit testing.
  • Developed Log4j logs to record the application events to provide the debugging, monitoring and auditing capabilities to the application.
  • Conducted code reviews and design reviews.
  • Developed the MAVEN, SBT scripts for preparing JAR files.
  • Collaborated with test teams, to support System and Integration Testing.

Environment: Java, J2EE, JDK1.7/1.8, Hadoop, HDP 2.2.4, HUE, YARN, Kafka, Spark, Scala, Mongo, HBase, REST, GIT, Junit, Maven, SBT, IntelliJ.

Confidential, NY

Hadoop and Senior Java Developer

Responsibilities:

  • Used Agile methodology in developing the application, which included iterative application development, weekly Sprints, stand up meetings and customer reporting backlogs.
  • Worked on a live 125 nodes Hadoop cluster running CDH4.4
  • Worked with highly unstructured and semi structured data of 90 TB in size (TB of data with replication factor of 3)
  • Extracted the data from Oracle into HDFS using Sqoop.
  • Created and worked SQOOP(version 1.3) jobs with incremental load to populate Hive External tables.
  • Extensive experience in writing Pig (version 0.11) scripts to transform raw data from several data sources into forming baseline data.
  • Developed Hive(version 0.10) scripts for end user / analyst requirements to perform ad hoc analysis
  • Very good understanding of Partitions, Bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance
  • Solved performance issues in Hive and Pig scripts with understanding of Joins, Group and aggregation and how does it translate to MapReduce jobs.
  • Developed UDFs in Java as and when necessary to use in PIG and HIVE queries
  • Very good experience with both MapReduce 1 (Job Tracker) and MapReduce 2 (YARN) setups
  • Very good experience in monitoring and managing the Hadoop cluster using Cloudera Manager.
  • Developed user interface using JSP, Tiles Framework, JQuery, HTML, CSS, JavaScript, AJAX and JSTL.
  • Worked on CORE JAVA (Collections of type Generics, Templates) and used the core packages.
  • Web services are developed based on the SOAP and REST using Apache CXF
  • Web services tested using SOAPUI tool .
  • Extensively used Hibernate in data access layer to access and update information in the database.
  • Implemented Spring Framework for the information flow between database and Model.
  • Developed Ant scripts to build and deploy the app on Web sphere Application Server.
  • Configured tiles-defs.xml file to configure the presentation views.
  • Used hibernate.config.xml file for configurations and mapping entries to access database.
  • Used struts-config.xml file for defining Mapping Definitions and Action Forward Definitions.
  • Created stored procedures, triggers, sequences and PL/SQL statements to interact with the database.
  • Developed Test cases, Test scenarios & Test scripts for Unit testing and Black box testing.
  • Developed Log4j logs to record the application events to provide the debugging, monitoring and auditing capabilities to the application.
  • Used Eclipse IDE for developing the application.
  • Used JDBC API and HQL (Hibernate Query Language) to query the Database System and used JDBC Thin Driver to connect to the database.
  • Conducted code reviews and design reviews.
  • Developed the MAVEN scripts for preparing EAR and WAR files used to deploy J2EE components
  • Collaborated with test teams, to support System and Integration Testing.

Environment: Java, J2EE, JavaBeans, Hadoop, CDH4.4, Nagios, PIG, REST, SOAP, Servlets 3.0, JSP 2.x, Hibernate 3.0, Spring 3.0, JMS, JavaScript, HTML, CSS, Oracle 10g, WebSphere 7, Windows 2000/XP, IBM RAD, CVS, Log4j, Junit, Maven.

Confidential, New Jersey

Java Developer

Responsibilities:

  • Used Agile methodology in developing the application, which included iterative application development, weekly Sprints, stand up meetings and customer reporting backlogs.
  • Developed the application using Software Development Life Cycle.
  • Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
  • Moving data from HDFS to RDBMS and vice-versa using SQOOP.
  • Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis
  • Installed and configured Hadoop cluster in Test and Production environments
  • Performed both major and minor upgrades to the existing CDH cluster
  • Implemented Commissioning and Decommissioning of new nodes to existing cluster.
  • Analyzing/Transforming data with Hive and Pig.
  • Developed user interface using JSP, Tiles Framework, JQuery, HTML, CSS, JavaScript, AJAX and JSTL.
  • Worked on CORE JAVA (Collections of type Generics, Templates) and used the core packages.
  • Implements MVC, Singleton, Factory, DAO, Value Object, session Facade, Data Access Object, Business Object paradigm
  • Extensively used Hibernate in data access layer to access and update information in the database.
  • Implemented Spring Framework for the information flow between database and Model.
  • Developed Ant scripts to build and deploy the app on Web sphere Application Server.
  • Configured tiles-defs.xml file to configure the presentation views.
  • Used hibernate.config.xml file for configurations and mapping entries to access database.
  • Used struts-config.xml file for defining Mapping Definitions and Action Forward Definitions.
  • Created stored procedures, triggers, sequences and PL/SQL statements to interact with the database.
  • Developed Test cases, Test scenarios & Test scripts for Unit testing and Black box testing.
  • Developed Log4j logs to record the application events to provide the debugging, monitoring and auditing capabilities to the application.
  • Used Eclipse IDE for developing the application.
  • Used JDBC API and HQL (Hibernate Query Language) to query the Database System and used JDBC Thin Driver to connect to the database.
  • Conducted code reviews and design reviews.
  • Developed the MAVEN scripts for preparing EAR and WAR files used to deploy J2EE components
  • Collaborated with test teams, to support System and Integration Testing.
  • Troubleshoot and fix systems as part of Production support.

Environment: Java, J2EE, Hadoop, PIG, Hive, Struts 2.x, Tiles, JavaBeans, Servlets 3.0, JSP 2.x, Hibernate 3.0, Spring 3.0, RESTful Web services, JavaScript, HTML, CSS, Oracle 10g, PL/SQL, WebSphere 7, Windows 2000/XP, IBM RAD, CVS, Log4j, Junit, ANT.

Confidential, Stamford, CT

Java Develepor

Responsibilities:

  • Developed the application using Software Development Life Cycle.
  • Technical Documents includes UML Class diagrams, Use Case diagrams and Sequence diagrams, State diagrams designed using Rational Rose software.
  • Developed web tier by using Servlets, JSP, Custom Tag Libraries, Tiles, Java Script, XHTML, XSLT and XML.
  • Analyzing business requirements and identifying system impacts.
  • Bug fixing, minor enhancements for the existing application and Production support
  • Handling tickets that are raised by user through JTrack(Ticket tracking tool provided by Confidential )
  • Communicating or co-coordinating with offshore team to fix the bugs in the application
  • Implemented JPA (ORM Mapping tool) framework to interact with the database to update, retrieve, insert and delete values effectively.
  • Involved in the architecture of the modules according to the J2EE standards (Client tier, Web tier, Business tier).
  • Developed Apache Ant script to create war/ear file and deployed on the application server.
  • Performed both Server side and Client side Validations.
  • Developed SQL queries for Database access.
  • Used Hibernate for mapping objects to the relational database.
  • Implemented Spring-ORM for Data Persistence, Which Integrates Hibernate with Spring.
  • Involved in unit testing and Integration Testing using Junit, and developed an Ant Build Script which executes all the Junit test classes and generates the test failure report.

Environment: JDK1.5, J2EE1.4, JSP 2.0, HTML, Java Script, PL/SQL, Rational Rose/UML, XML, SOAP,Spring 2.5, JPA, Eclipse Indigo, Web logic 10, AJAX, JUNIT, Oracle 10g, FileNet, Linux.

Confidential, Stamford, CT

Java Develepor

Responsibilities:

  • Gathered the requirements and designed the application flow for the application.
  • Web services are developed based on the SOAP and REST using Apache CXF
  • Developed business controller layer using with DROOLS API, JAXB parser and Apache POI.
  • Web services tested using SOAPUI tool .
  • Used Spring Framework to integrate with Hibernate JPA.
  • Extensively worked with Hibernate JPA to connect to database for data persistence.
  • Used Log4J log both User Interface and Domain Level Messages.
  • Performed Unit testing for modules using Junit.
  • Played an active role in preparing documentation for future reference and upgrades.

Environment: Java 1.6, JPA, Spring 2.5, Hibernate, Eclipse, WebLogic 9, Oracle 9i, Junit, Log 4j, Apache CXF, JAXB, Apache POI and DROOLS API.

Confidential

Java Developer

Responsibilities:

  • Design and development of User Interfaces using JSP, XML, HTML, AJAX and CSS.
  • Developed Servlets, JSP pages and Session Beans.
  • Used JSP extensively to display and manipulate data.
  • Created tables in the data base using SQL.
  • Interpreted and manipulated Struts and hibernate configure files.
  • Developed different JSP pages (GUI) & client side user validations using JavaScript DOJO.
  • Used Session Facade, Data Access Object, Business Object and Value Object design patterns.
  • Created JUnit test suite to automate the testing of the business logic

Environment: JDK, Struts, XML, JSP, Servlets, Hibernate, Maven,Java Script, CSS, UML, Eclipse, Jboss Application Server, MySql, JUNIT,Linux

We'd love your feedback!