Hadoop Developer Resume
Duluth, GA
SUMMARY:
- Over 8 years of professional IT experience and proven expertise in complete SDLC life cycle comprising of 4 years hands on experience in Hadoop Eco system technologies and 4 years of Web - based and Client-Server business applications with emphasis on Object Oriented Java / J2EE technologies.
- Extensive knowledge and experience in Hadoop Architecture, and its Eco-system technologies to build high-volume real-time data processing applications.
- Experienced building scalable and distributed data processing applications on HDFS, using Hive, MapReduce, sqoop, flume, Pig, and Oozie.
- Hands on experience with migrating data from different Relational Database Systems to HDFS using Hive, Sqoop and vice versa.
- Involved in complete flow of the application, starting from data ingestion from upstream to HDFS, processing the data in HDFS and analyzing data.
- Worked on developing Enterprise applications using Java and various J2EE technologies including Hibernate3.x/4.0, Spring MVC, JSP, JavaScript, JQuery, XML, HTML, SOA, Web services like SOAP, REST, UML, Design/Architectural patterns. Experience in application development and deployment using application servers as Apache Tomcat6.0/7.0 and WebLogic.
- Extensive experience in all phases of SDLC and Agile Methodology.
- Experience on Eclipse IDE 4.3/4.4/4.6, IntelliJ 14.1.
- Worked on Distributed cluster setup and configuration, Data Analysis.
- Experience working with different databases, such as Oracle 11g/10g/9i, MySQL, SQL Server 2008, 2012 and writing stored procedures, functions, joins, and triggers.
- Performed unit testing using JUnit, Integration and deployment of applications done using tools like ANT and Maven and debugging, logging through log4j.
- Experienced in developing UML diagrams like use case, class and sequence diagrams using MS Visio.
- Proactive, highly motivated, organized and result oriented with excellent interpersonal skills. Strong verbal and written communication skills. Self-motivating and be able to fit in a team with diverse professionals.
TECHNICAL SKILLS:
- Java
- C
- C++
- Apache Hadoop 1.0/2.x
- HDFS
- MapReduce
- Hive 1.2
- Sqoop
- Flume1.4/1.5
- HBase
- RDBMS
- NoSQL
- MongoDB 2.4/3.x
- Spark 2.1
- SQL server 2008/2012
- PL/SQL
- Oracle 11g/10g
- MySQL
- DB2
- Big Data
- PIG
- Oozie Workflow
- ETL
- Talend
- Tableau
- Cloudera
- Hortonworks
- Apache Kafka
- Zookeeper
- JSP
- JMS
- Grails 2.3/Groovy
- Web Services
- SOAP
- REST
- XML
- XSLT
- JAXB
- Ambari
- Hibernate 3.x/4.2
- Scala
- Spring 2.5/3.2
- Spring - Boot
- Java Script
- HTML
- CSS
- AngularJS
- JQuery
- ActiveMQ
- Mule ESB 3.3
- WebLogic
- Apache Tomcat 6.0/7.0
- Shell scripting
- MS-Access
- MS-Excel
- SDLC
- Agile
- Scrum
- ANT
- Maven
- CVS log4j
- JUnit
- UML
- Microsoft Visio
- IBM Rational Rose
- SQL Developer
- FileZilla
- Eclipse IDE
- IntelliJ
- GIT
- Putty
- Jira
- Jenkins
- Windows
- UNIX
- LINUX.
PROFESSIONAL EXPERIENCE:
Hadoop Developer
Confidential, Duluth, GA
Responsibilities:- Worked in an agile development environment, evaluated business requirements and prepare the business requirements and design documents.
- Participate in Design reviews and daily project scrums.
- Interacted with business clients and data source teams gathering business process and technical details about how data was generated.
- Imported data from various data sources, performed transformations using Hive, Pig and loaded data into HDFS for aggregations.
- Ingested the raw data into HDFS in batch mode using sqoop and SFTP with FS Shell. Setup and configured Flume for real time data ingestion.
- Performed Hive partitioning, bucketing and executing different types of joins on Hive tables and implementing Hive servers like JSON and Avro.
- Experience with NoSQL databases like Hbase, and MongoDB.
- Worked on Zookeeper and Oozie operational Service for coordinating the cluster and scheduling workflows.
- Written Sqoop incremental import job to move new / updated info from Database to HDFS.
- Developed MapReduce jobs for log analysis Analytics to generate reports on failovers.
- Used Oozie workflow engine to manage interdependent Hadoop jobs and to automate several types of Hadoop jobs such as Java MapReduce, Hive, Sqoop and spark using scala.
- Experience in HBase database manipulation with structured, unstructured and semi-structured datasets.
- Worked on extract, transform, and load (ETL) data from multiple data sources.
- Generated various marketing reports using Tableau with Hadoop as a source for data.
- Experienced in writing Unix Shell Scripts for file backup.
- Developed Presentation tier as HTML, Java Server Pages using Struts MVC Framework
- Extensive Business knowledge related to Healthcare industry and its terms HIPAA, ICD, ANSI EDI, DME, TT, PT, etc. Ancillary services with Insurance claims process.
- Wrote unit and integration test cases.
Environment: Apache Hadoop, MapReduce, Java, HDFS, HIVE, Sqoop, Spark, SQL Server 2012, Talend, JUnit, log4j, JavaScript, XML, Eclipse, Putty, GIT, Jenkins, Maven, Windows, Linux.
Hadoop Developer
Confidential, Columbus, OH
Responsibilities:- Developing multiple MapReduce programs in java for data extraction, transformation and aggregation from multiple file formats including XML, JSON, CSV and other file formats.
- Developed Java programs to process huge JSON files received from marketing team to convert into format standardized for the application.
- Proof of concept on spark for integration source transformations.
- Used Spark API over Cloudera Hadoop YARN to perform analytics on data.
- Exploring with the Spark improving the performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frame, Spark YARN.
- Developed Pig Scripts, Pig and Hive UDFs to process the data.
- Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs.
- Applied optimizations on Hive tables for faster querying as well as performance tuning techniques on MapReduce and Pig jobs to improve running time
- Used Oozie workflow engine to manage interdependent Hadoop jobs and to automate several types of Hadoop jobs such as map-reduce Hive, Pig, and Sqoop.
- Creating the cube in talend to create different types of aggregation in the data and also to visualize them.
- Generated various marketing reports using Tableau and Spotfire.
- Conduct/Participate in project team meetings to gather status, discuss issues and action items.
- Communicate deliverables status to user/stakeholders, client and drive periodic review meetings.
Environment: Hadoop, Spark, MapReduce, Hive, Sqoop, Flume, Oozie, Python, Splunk, Shell scripting, SVN, Linux, Putty, Oracle 11g, Tableau.
Hadoop Developer
Confidential, Duluth, GA
Responsibilities:- Implemented Flume to import log data from web server into HDFS.
- Worked with large scale distributed data solution Cloudera CDH4 cluster.
- Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
- Creation of Design Documents, System Test Cases, Unit Test Cases and review document, Migration Documents.
- Monitored Map Reduce Programs which are running on the cluster.
- Worked on Git for version control, JIRA for project tracking and Jenkins for continuous Integration.
- Involved in development, design and implementation front-end part of the widget-based application using HTML, CSS, JQuery, JavaScript.
- Used Ajax, JSON with JQuery for request data and response processing.
- Experience in application performance tuning and troubleshooting.
- Developed Presentation tier as HTML, Java Server Pages using Struts MVC Framework
Environment: Apache Hadoop, HBase, Hive, Oozie, JSON, SQL, AngularJS, XML, Java, HTML, CSS.
Java/ J2EE Developer
Confidential, Dallas, TX
Responsibilities:- Involved in all the phases of SDLC including Requirements Collection, Design & Analysis of the Customer Specifications, and Development & Customization of the Application.
- Prepared Use case, Class and Sequence diagrams using Rational Rose tool.
- Developed the application under J2EE architecture using JSP, Struts, Java Beans, iBATIS Data mapper.
- Developed many JSP pages, used Dojo in JavaScript Library, jQuery UI for client side validation. Client & server validation were handled using Struts validator.
- Developed UI using JSP, Controller using Struts Action Class, Model using Java Beans as POJO.
- Implemented Java Message Services (JMS) using JMS API.
- Performed Unit testing, System Testing and Integration Testing.
- Prepared the system test plan and test cases to suit business requirements and system specification documents.
- Used Maven to compile and generate EAR, WAR & JAR. Used Log4j for logging Errors.
- Used SVN for version control & source code management.
Environment: JDK 1.5, J2EE 1.4, JSP, Struts 1.3, Struts Tiles, validator, EJB 2.0 (Session, MDB, JMS), Hibernate 3.3, XML, UML, Oracle, BEA WebLogic Server 9.1, Eclipse 3.2, Ajax, Ant, JUnit, Log4j, Maven 1.9, CVS, Rational Rose, JavaScript, Red Hat Linux.
Java/ J2EE Developer
Confidential, Monroe, LA
Responsibilities:- Involved in translating functional requirements into technical requirements.
- Experience in document analysis and technical feasibility discussions for implementing new functionalities.
- Applied design patterns including MVC Pattern, Facade Pattern, Abstract Factory Pattern, DAO Pattern and Singleton.
- Developed front end screens using JSP, Struts View Tags, XLTS, DHTML, HTML5, CSS3, JavaScript, and spring.
- Used Exception Handling, STRUTS Validator Framework. And with Strong knowledge in Spring Framework using IOC/AOP, Spring Transaction support (Declarative/ Programmatic), Hibernate in spring.
- Involved in server side and front-end validation using Struts Validation framework and JavaScript.
- Developed various Database interaction objects by implementing the DAO patterns.
- Generated Spring XML files for the configured beans.
- Used Hibernate for mapping POJO’s to relational database tables using xml files
- Used SAX Parser for parsing the xml document
- Involved in Unit Testing and Bug-Fixing and achieved the maximum code coverage using JUNIT test cases .
Environment: Core Java, Java 1.4, JSP, HTML5, JavaScript, Struts 1.2, Hibernate 3.0, Spring 2.0, JSF, JMS, ANT, AJAX, Design Patterns, Servlets, Struts Tag Libraries/JSTL, XML, UML, JUnit, Oracle 10g, SVN, Web Services, Agile, Log4J, CSS, Windows XP.
Software Engineer
Confidential
Responsibilities:- Involved with interacting with the clients and application user for their Requirements, Specifications and enhancements.
- Involved in design and development of Servlets and JSPs using Apache Struts framework.
- Used JDBC, Data Sources and Connection Pooling in Application server.
- Implemented J2EE Design Patterns such as Session Facade to reduce the Network Traffic and Service Locator.
- Designed and developed a user usage logging facility using Apache Log4J.
- Implemented Complete client side validations in JavaScript.
- Used ANT to write build scripts as well as deployment scripts.
- Packed and deployed the entire application code to integration testing environment for all the releases.
- Involved in JUnit tests for the services and documented the services developed.
- Provided production support by interacting with the end-users and fixing bugs.
Environment: Java, J2EE, Struts 1.1, LINUX, JSP/Servlets, CSS, WebLogic 9.x, Eclipse 3.0, JDBC, XML, HTML, Oracle 9i, UML, JUnit, SVN, ANT 1.3/1.4, SOAP, Web Services.