Hadoop/spark Developer Resume
PhiladelphiA
SUMMARY
- Seasoned Java/Hadoop/Spark/Scala developer with 7+ years of experience in Object Oriented programming.
- 5+ Years hands on experience on Java/J2EE technologies
- 2.6+ years hands on experience on Hadoop (HDFS, Map Reduce, PIG, HIVE, and SQOOP).
- 1+ year’s hands on experience on Spark (1.5, 1.6) & Scala Full stack developer.
- Work experience on top different domains Banking, Financial, Health Care, Ecommerce and Media.
- Worked on a live 480 nodes Hadoop cluster running Horton Works and CDH4.4
- Good knowledge on Akka, Angular JS, Node JS, React JS, NoSQL (Mongo DB, Cassandra) and Kafka
- Moving data from HDFS to RDBMS and vice - versa using SQOOP, Spark.
- Good knowledge on HBase, OOZIE, Mongo DB, Cassandra, HUE, SPLUNK, YARN.
- Experience in using Text, Parquet, ORC, Sequence files, AVRO file formats.
- Proficient in the development of client/server and web Applications using JAVA, J2EE, J2SE technologies and Web Technologies for enterprise applications.
- Expert in developing enterprise web-based application based on Java technologies using J2EE, J2SE, Java Servlets, Springs, Struts, Java Beans, JMS, JNDI, ANT and Apache Maven.
- Expertise in developing web based GUIs using Java Applets, HTML, DHTML, JavaScript, XML, DOJO, AJAX, JSP, EXT-Js, Angular JS.
- Good working knowledge on versioning tools CVS, GIT etc...
- Hands on experience in configuration and deployment of multi-tier applications using servers like WebSphere, Web Logic and JBoss.
- Extensive experience to SOA (Service Oriented Architecture) and experience in developing Web services REST and SOAP using Apache CXF, WSDL, SOAP, WS Security, Apache POI, XML-RPC and JAXB/JAXP.
- Extensive experience working in Spring 2.5/3.0 framework, Spring batch, Struts 1.0/1.1/2.x framework, O/R Mapping Hibernate 3.x framework.
- Strong experience in writing SQLs, Stored Procedures and Triggers.
- Working experience in relational databases like ORACLE, Tera Data and SQL Server.
- Good Working Knowledge in Ant, Maven, SBT for project build/test/deployment, Log4j for logging and JUnit for unit and integration testing.
- Strong knowledge of IDEs like Eclipse, IBM Rational Application Developer (RAD 7.0.0) IntelliJ.
- Extensive experience in software development Agile methodologies.
- Excellent team player with leadership abilities. Strong analytical skills with excellent interpersonal and communication skills with analytical ability and understanding of Organizational functioning.
TECHNICAL SKILLS
Programming Technologies: JDK1.6/1.7/1.8, Scala, Akka, JSP 1.2/2, Servlets, Ajax, JSF, Hibernate3.0, Spring2.X, Spring3.X, JDBC, JUnit, JNDI, XML, JMS, Web Services, REST, SOAP,JSP,JSTL,XSLT
Big Data/Distributed Data Management: Spark, Hadoop, Map Reduce, YARN, HIVE, PIG, HBASE, SQOOP, HDP (Horton), CDH, Kafka
UI Technologies: HTML, Java Script, DOJO, Ajax,JSON, CSS, JQuery, Angular JS, Node JS, React
Web/App Servers: IBM Websphere 5.1/6.x, Bea Web Logic Server 9.2, Tomcat 4.0/5.0, Apache1.3/2.0, JBoss 4.x
RDBMS: Oracle 10g, 11g, Mysql, TeraData
Operating System: Windows XP/7/10, Linux, Unix
Development Tools: Eclipse, IntelliJ, RAD6.x, 7.x, Jet Brains Web Storm, IBM WebSphere 5.x/6.x, IBM MQ Series, TOAD, ANT, MAVEN, SQL Loader, IBM Messaging Queue series(MQ).
PROFESSIONAL EXPERIENCE
Confidential, Philadelphia
Hadoop/Spark Developer
Responsibilities:
- Used Agile methodology in developing the application, weekly Sprints, stand up meetings.
- Worked on a live 480 nodes Hadoop cluster running on HDP 2.2.4
- Worked with highly unstructured and semi structured data of 2PB in size (TB of data with replication factor of 3)
- Developed Spark Jobs pipeline which are used to stream, transformations, aggregations and generating the daily snapshots of customer data.
- Developed UI component and REST web services using React JS and Scala
- Developed Map Reduce Jobs (using Deprecated and MR2) to generate the Flat files on data from multiple sources like hive table, text files data.
- Developed MR code in Eclipse IDE using JAVA, Multiple Inputs, Multiple Outputs, Aggregations, and Joins.
- Extracted the RDBMS (Oracle, TERADATA etc..)Data into HDFS and Hive table using SQOOP and Shell scripts.
- Created and worked SQOOPjobs with incremental load to populate Hive External tables.
- Developed Hive(version 0.14) scripts and written HQL’s for ETL process on data.
- Very good understanding of Partitions, Bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance
- Worked on posting messages on Kafka (Producer and Consumer) topics.
- Developed REST service to post the messages on Kafka topics.
- Working experience with both MapReduce 1 (Job Tracker) and MR2 (YARN)
- MR jobs and Work flow jobs monitoring and managing using HUE and UC4.
- Developed Test cases, Test scenarios & Test scripts for Unit testing.
- Developed Log4j logs to record the application events to provide the debugging, monitoring and auditing capabilities to the application.
- Conducted code reviews and design reviews.
- Developed the MAVEN, SBT scripts for preparing JAR files.
- Collaborated with test teams, to support System and Integration Testing.
Environment: Java, J2EE, JDK1.7/1.8, Hadoop, HDP 2.2.4, HUE, YARN, Kafka, Spark, Scala, Mongo, HBase, REST, GIT, Junit, Maven, SBT, IntelliJ.
Confidential, NY
Hadoop and Senior Java Developer
Responsibilities:
- Used Agile methodology in developing the application, which included iterative application development, weekly Sprints, stand up meetings and customer reporting backlogs.
- Worked on a live 125 nodes Hadoop cluster running CDH4.4
- Worked with highly unstructured and semi structured data of 90 TB in size (TB of data with replication factor of 3)
- Extracted the data from Oracle into HDFS using Sqoop.
- Created and worked SQOOP(version 1.3) jobs with incremental load to populate Hive External tables.
- Extensive experience in writing Pig (version 0.11) scripts to transform raw data from several data sources into forming baseline data.
- Developed Hive(version 0.10) scripts for end user / analyst requirements to perform ad hoc analysis
- Very good understanding of Partitions, Bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance
- Solved performance issues in Hive and Pig scripts with understanding of Joins, Group and aggregation and how does it translate to MapReduce jobs.
- Developed UDFs in Java as and when necessary to use in PIG and HIVE queries
- Very good experience with both MapReduce 1 (Job Tracker) and MapReduce 2 (YARN) setups
- Very good experience in monitoring and managing the Hadoop cluster using Cloudera Manager.
- Developed user interface using JSP, Tiles Framework, JQuery, HTML, CSS, JavaScript, AJAX and JSTL.
- Worked on CORE JAVA (Collections of type Generics, Templates) and used the core packages.
- Web services are developed based on the SOAP and REST using Apache CXF
- Web services tested using SOAPUI tool .
- Extensively used Hibernate in data access layer to access and update information in the database.
- Implemented Spring Framework for the information flow between database and Model.
- Developed Ant scripts to build and deploy the app on Web sphere Application Server.
- Configured tiles-defs.xml file to configure the presentation views.
- Used hibernate.config.xml file for configurations and mapping entries to access database.
- Used struts-config.xml file for defining Mapping Definitions and Action Forward Definitions.
- Created stored procedures, triggers, sequences and PL/SQL statements to interact with the database.
- Developed Test cases, Test scenarios & Test scripts for Unit testing and Black box testing.
- Developed Log4j logs to record the application events to provide the debugging, monitoring and auditing capabilities to the application.
- Used Eclipse IDE for developing the application.
- Used JDBC API and HQL (Hibernate Query Language) to query the Database System and used JDBC Thin Driver to connect to the database.
- Conducted code reviews and design reviews.
- Developed the MAVEN scripts for preparing EAR and WAR files used to deploy J2EE components
- Collaborated with test teams, to support System and Integration Testing.
Environment: Java, J2EE, JavaBeans, Hadoop, CDH4.4, Nagios, PIG, REST, SOAP, Servlets 3.0, JSP 2.x, Hibernate 3.0, Spring 3.0, JMS, JavaScript, HTML, CSS, Oracle 10g, WebSphere 7, Windows 2000/XP, IBM RAD, CVS, Log4j, Junit, Maven.
Confidential, New Jersey
Java Developer
Responsibilities:
- Used Agile methodology in developing the application, which included iterative application development, weekly Sprints, stand up meetings and customer reporting backlogs.
- Developed the application using Software Development Life Cycle.
- Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
- Moving data from HDFS to RDBMS and vice-versa using SQOOP.
- Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis
- Installed and configured Hadoop cluster in Test and Production environments
- Performed both major and minor upgrades to the existing CDH cluster
- Implemented Commissioning and Decommissioning of new nodes to existing cluster.
- Analyzing/Transforming data with Hive and Pig.
- Developed user interface using JSP, Tiles Framework, JQuery, HTML, CSS, JavaScript, AJAX and JSTL.
- Worked on CORE JAVA (Collections of type Generics, Templates) and used the core packages.
- Implements MVC, Singleton, Factory, DAO, Value Object, session Facade, Data Access Object, Business Object paradigm
- Extensively used Hibernate in data access layer to access and update information in the database.
- Implemented Spring Framework for the information flow between database and Model.
- Developed Ant scripts to build and deploy the app on Web sphere Application Server.
- Configured tiles-defs.xml file to configure the presentation views.
- Used hibernate.config.xml file for configurations and mapping entries to access database.
- Used struts-config.xml file for defining Mapping Definitions and Action Forward Definitions.
- Created stored procedures, triggers, sequences and PL/SQL statements to interact with the database.
- Developed Test cases, Test scenarios & Test scripts for Unit testing and Black box testing.
- Developed Log4j logs to record the application events to provide the debugging, monitoring and auditing capabilities to the application.
- Used Eclipse IDE for developing the application.
- Used JDBC API and HQL (Hibernate Query Language) to query the Database System and used JDBC Thin Driver to connect to the database.
- Conducted code reviews and design reviews.
- Developed the MAVEN scripts for preparing EAR and WAR files used to deploy J2EE components
- Collaborated with test teams, to support System and Integration Testing.
- Troubleshoot and fix systems as part of Production support.
Environment: Java, J2EE, Hadoop, PIG, Hive, Struts 2.x, Tiles, JavaBeans, Servlets 3.0, JSP 2.x, Hibernate 3.0, Spring 3.0, RESTful Web services, JavaScript, HTML, CSS, Oracle 10g, PL/SQL, WebSphere 7, Windows 2000/XP, IBM RAD, CVS, Log4j, Junit, ANT.
Confidential, Stamford, CT
Java Develepor
Responsibilities:
- Developed the application using Software Development Life Cycle.
- Technical Documents includes UML Class diagrams, Use Case diagrams and Sequence diagrams, State diagrams designed using Rational Rose software.
- Developed web tier by using Servlets, JSP, Custom Tag Libraries, Tiles, Java Script, XHTML, XSLT and XML.
- Analyzing business requirements and identifying system impacts.
- Bug fixing, minor enhancements for the existing application and Production support
- Handling tickets that are raised by user through JTrack(Ticket tracking tool provided by Confidential )
- Communicating or co-coordinating with offshore team to fix the bugs in the application
- Implemented JPA (ORM Mapping tool) framework to interact with the database to update, retrieve, insert and delete values effectively.
- Involved in the architecture of the modules according to the J2EE standards (Client tier, Web tier, Business tier).
- Developed Apache Ant script to create war/ear file and deployed on the application server.
- Performed both Server side and Client side Validations.
- Developed SQL queries for Database access.
- Used Hibernate for mapping objects to the relational database.
- Implemented Spring-ORM for Data Persistence, Which Integrates Hibernate with Spring.
- Involved in unit testing and Integration Testing using Junit, and developed an Ant Build Script which executes all the Junit test classes and generates the test failure report.
Environment: JDK1.5, J2EE1.4, JSP 2.0, HTML, Java Script, PL/SQL, Rational Rose/UML, XML, SOAP,Spring 2.5, JPA, Eclipse Indigo, Web logic 10, AJAX, JUNIT, Oracle 10g, FileNet, Linux.
Confidential, Stamford, CT
Java Develepor
Responsibilities:
- Gathered the requirements and designed the application flow for the application.
- Web services are developed based on the SOAP and REST using Apache CXF
- Developed business controller layer using with DROOLS API, JAXB parser and Apache POI.
- Web services tested using SOAPUI tool .
- Used Spring Framework to integrate with Hibernate JPA.
- Extensively worked with Hibernate JPA to connect to database for data persistence.
- Used Log4J log both User Interface and Domain Level Messages.
- Performed Unit testing for modules using Junit.
- Played an active role in preparing documentation for future reference and upgrades.
Environment: Java 1.6, JPA, Spring 2.5, Hibernate, Eclipse, WebLogic 9, Oracle 9i, Junit, Log 4j, Apache CXF, JAXB, Apache POI and DROOLS API.
Confidential
Java Developer
Responsibilities:
- Design and development of User Interfaces using JSP, XML, HTML, AJAX and CSS.
- Developed Servlets, JSP pages and Session Beans.
- Used JSP extensively to display and manipulate data.
- Created tables in the data base using SQL.
- Interpreted and manipulated Struts and hibernate configure files.
- Developed different JSP pages (GUI) & client side user validations using JavaScript DOJO.
- Used Session Facade, Data Access Object, Business Object and Value Object design patterns.
- Created JUnit test suite to automate the testing of the business logic
Environment: JDK, Struts, XML, JSP, Servlets, Hibernate, Maven,Java Script, CSS, UML, Eclipse, Jboss Application Server, MySql, JUNIT,Linux
