Sr. Hadoop Developer Resume
Miamisburg, OH
SUMMARY
- 8+ years of extensive IT experience with multinational clients this includes 3 years of Hadoop related experience developingBig Data / Hadoop applications.
- Experience with installation, configuration, supporting and managing of Big Data and underlying infrastructure of Hadoop Cluster.
- Extensive knowledge in programming and software development experience with skills in data analysis, design, development, testing and deployment of software systems from development stage to production stage with giving emphasis on Object oriented paradigm.
- Experience in XML, HTML, Tag Libraries, Custom Tags, and Design Patterns.
- Hands on experience in installing configuring and using Hadoop ecosystem components like HadoopMapReduce, HDFS, HBase, Hive, Sqoop, Pig, Oozie, Zookeeper and Flume with CDH4&5 distributions and EC2 cloud computing with Amazon Web Services (AWS).
- Good Exposure on Apache HadoopMapReduce framework, PIG Scripting and HDFS.
- Excellent knowledge on Hadoop ecosystems such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, YARN and Map Reduce programming paradigm.
- Experience in analyzing data using HiveQL, Pig Latin, HBase and custom Map Reduce programs in Java.
- Extending HIVE and PIG core functionality by using custom UDFs and have a very good working knowledge on Pentaho ETL, IBM Big Insight and Cassandra.
- Good understanding of Talend ETL.
- Have knowledge on Talend Open Studio.
- Experience with all flavors of Hadoop distributions, which includes Cloudera, Hortonworks and MapR.
- Good understanding of NoSQL databases and hands on experience in writing applications on NoSQL databases like HBase and Cassandra.
- Database Developer: Hands on experience with Database Design, SQL and PL/SQL.
- Strong development experience in Message Oriented and Service Oriented Technologies like WSDL/SOAP and SOA (Web Services) and RESTful API.
- Excellent working knowledge in Application servers such as WebSphere, WebLogic, JBoss, Apache Tomcat.
- Experience in exporting/importing the data using Sqoop from HDFS to Relational Database systems/mainframe and vice - versa. Used Hadoop Streaming utility well to run MapReduce jobs.
- Experienced in Core Java, Servlets, JSP, Struts, Spring, Hibernate, JDBC and Web Service.
- Experience in Design and Development of java web services using XML, SOAP, WSDL, and UDDI based on SOA and has excellent understanding of xml technologies XML, XSD, XSL, SAX, DOM, JAXB 2.0.
- Experience in developing HTML/PHP/JSP templates using Adobe Dreamweaver.
- Have knowledge on developingHadoopstreaming MapReduce works usingPython.
- Experienced in all facets of Software Development Life Cycle (Analysis, Design, Development, Testing and Maintenance) using Waterfall and Agile methodologies.
- Extensively worked in Agile, TDD and Scrum development methodology.
TECHNICAL SKILLS
Big Data: Hadoop, HDFS, MapReduce, Hive, Sqoop, Pig, HBase, NoSQL, Flume, Zookeeper, Oozie, Impala.
Web Technologies: Java, J2EE, JSP, Servlets, Struts, Hibernate, Spring, Spring MVC, Spring DAO, Spring Security, RMI, JDBC, JMS, DHTML, XML, XSLT, Spring WS, Drools, Jboss Enterprise portal, Jboss seams, JSTL, EJB, Web Services, JSF, Rich Faces, Birt Report, Crystal Reports, HTML 5, CSS, Ajax, SOAP, JavaScript, Web Services.
Languages: Java, PL/SQL.
Frameworks: Hadoop, HDFS, Map Reduce, Pig, Hive.
Databases: SQL Server, MySQL, DB2, Oracle.
Operating Systems: Windows, Linux, Unix (Sun Solaris), Ubuntu.
Version Control: Github, SVN.
Development Tools: Eclipse, SOAP UI, HP QTP, File Aid, QMF, Spufi, Visual Source Safe, ENDEVOR, XPEDITOR, Test Director, Team Forge.
Other Tools: SQL Developer, Maven, ANT, Log4J, Junit.
Domain Knowledge: Health Care, Retail, People Systems, Finance.
PROFESSIONAL EXPERIENCE
Confidential, Miamisburg, OH
Sr. Hadoop developer
Responsibilities:
- Worked on the proof-of-concept for Apache Hadoop framework initiation.
- Developed complex Map Reduce programs in Java for Data Analysis on different data formats.
- Developed Map Reduce programs that filter bad and un-necessary records and find out unique records based on different criteria.
- Developed Secondary sorting implementation to get sorted values at reduce side to improve map reduce performance.
- Worked on documentation of all Extract, Transform and Load, designed, developed, validated and deploy the Talend ETL processes for Data ware house team using PIG, HIVE on Hortonworks Hadoop.
- Optimizing Talend jobs.
- Loading log data into HDFS using Flume, Kafka.
- Implemented custom Data Types, Input Format, Record Reader, Output Format, Record Writer for Map Reduce computationsto handle custom business requirements.
- Experience with distributed systems, map reduce systems, data modeling and Big Data systems
- Responsible for performing extensive data validation using Hive.
- Implemented Map Reduce programs to classified data organizations into different classifieds based on different type of records.
- Worked on Sequence files, RC files, Map side joins, bucketing, partitioning for Hive performance enhancement and storage improvement.
- Implemented Daily Oozi jobs that automate parallel tasks of loading the data into HDFS and pre-processing with Pig using Oozie co-coordinator jobs.
- Importing and exporting data into HDFS and Hive using Sqoop.
- Perform data analysis using Hive and Pig.
- Worked intuning Hive and Pig scriptsto improve performance.
- Experience with spark for streaming data analysis.
- Knowledge on handling Hive queries using Spark SQL that integrate Spark environment.
- Involved in submitting and tracking Map Reduce jobs using JobTracker.
- Involved in loading the created HFiles into HBase for faster access of large customer base without taking Performance hit.
- Implemented Hive Generic UDF's to implement business logic.
- Involved in creating Distribute cache, join and filtering.
- Configured build scripts for multi module projects with Maven.
Environment: HDFS, Map Reduce, Hive,Kafka, Sqoop,Spark, Flume, Zookeeper, Oozie, HFiles, HBase, Hadoop, CDH4, Java, Linux, Maven, Oracle 11g/10g, SVN, JDK 1.7, JSP, Agile, ETL, Crunch API, HTML, XML, JavaScript, Toad 9.6, UNIX Shell Scripting.
Confidential, Chicago, IL
Sr. Hadoop Developer
Responsibilities:
- Worked on the proof-of-concept for Apache Hadoop framework initiation.
- Experience in HDFS, MapReduce and Hadoop Framework.
- Trained and guided the team on Hadoop framework, HDFS, MapReduce concepts.
- Configured the Hadoop Cluster in Local (Standalone), Pseudo-Distributed, Fully-Distributed Mode.
- Developed MapReduce jobs for Log Analysis, Recommendation and Analytics.
- Wrote MapReduce jobs to generate reports for the number of activities created on a particular day, during a dump from the multiple sources and the output was written back to HDFS.
- Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
- Development of Hadoop Map Reduce programs.
- Experience in writing MapReduce jobs in python for some complicated queries.
- Moving all the log information into HDFS.
- Retrieved data using HQL from Hive.
- Identifying insurances are referred to customers.
- Grouping the same insurances by analyzing the messages.
- Written Map Reduce code to convert semi Structured Data to Structured data.
- Developed a Framework that will create external and manageable tables in a batch processing based on the metadata files.
- Analyzed data using RStudio.
- Successfully designed and developed a solution for speeding up a SQL Job using Hadoop Map-Reduce framework. Processing time was reduced from hours to Minutes.
- Involved in migrating the data from development cluster to QA cluster and from there to production cluster.
- Created the developer Unit test plans and executed unit testing in the development cluster.
Environment: JDK 1.6, Struts 1.3, JSP, Agile, ETL, Crunch API, HTML, JavaScript, Hadoop distribution of, Cloudera, Shell, Linux, Pig, Hive HQL, MapReduce, HBase, Sqoop, Oozie, Ganglia and Flume.
Confidential, San Ramon, CA
Hadoop / Sr. java Developer
Responsibilities:
- Developed multiple MapReduce jobs in Java for data cleaning and preprocessing.
- Importing and exporting data into HDFS and Hive using Sqoop.
- Experienced in defining job flows.
- Experienced in managing and reviewing Hadoop log files.
- Experienced in running Hadoop streaming jobs to process terabytes of xml format data.
- Load and transform large sets of structured, semi structured and unstructured data.
- Responsible to manage data coming from different sources.
- Supported Map Reduce Programs those are running on the cluster.
- Involved in loading data from UNIX file system to HDFS.
- Installed and configured Hive and also written Hive UDFs.
- Involved in creating Hive tables, loading data and writing Hive queries which will run internally in map reduce way.
- Gained very good business knowledge on health insurance, claim processing, fraud suspect identification, appeals process etc.
Environment: Hadoop, MapReduce, HDFS, Hive, Java (jdk1.6), Hadoop distribution of Cloudera, DataStax, Flat files, Oracle 11g/10g, PL/SQL, SQL PLUS, UNIX Shell Scripting, Autosys r11.0.
Confidential, Atlanta, GA
Sr. Java Developer
Responsibilities:
- Worked on Gradle build tool for build the war file for Customer Gateway.
- Used JBoss EAP6.0.1 for deploy and configured the CG application.
- Used Confluence repository for saving Customer Gateway documents and files.
- Worked on Rest API and Customer Gateway application uses HTTP basic authentication for its entire set of APIs.
- Developed Restful Web service to expose recent activity of the user as service using Jersey framework.
- Moved current functionality that was connecting to the database using JPA to access data through REST calls.
- Worked on JDBC for create/close database connections.
- Worked on Rest client that uses HTTP client.
- Socket programming experience with python .
- Used application/xml for API supports both inbound and outbound response data bindings.
- Worked on SOAP1.2 web services for consume and produce external system uses cox communication.
- Worked on Collection framework (Map/List) to set and get the query Params (CG).
- Worked on ORACLE 10g for storing and retrieving the data from database.
- Worked on UNIX machines for Deploy/Configure the JBoss EAP server to build the CG application.
- Worked on Server Tuning for increase the heap size for UNIX machines.
- Handled offshore team for implement Customer gateway design/architecture.
- Worked on Pl/SQL for query and fetch the data from database.
- Implemented web layer using JSF and Ice faces.
- Implemented business layer using Spring MVC.
- Implemented Getting Reports based on start date using HQL.
- Implemented Session Management using Session Factory in Hibernate.
- Developed the DO’s and DAO’s using hibernate.
- Implement SOAP web service to validate zip code using Apache Axis.
- Wrote complex queries, PL/SQL Stored Procedures, Functions and Packages to implement Business Rules.
- Wrote PL/SQL program to send EMAIL to a group from backend.
Environment: Core Java, Python, J2EE, Log4J, JUnit, JSF, Git, SOA, SQL, REST, JIRA, Apache Tomcat, JSP, JSTL, CSS, GWT, CVS, Servlets, Struts, DB2, PL/SQL, Oracle JDBC,MVC, HTML, DHTML,Javascript, AJAX, JQUERY, Web Services, Hibernate, JBoss EAP 6.0.1, Oracle10g, UNIX.
Confidential
Java Developer
Responsibilities:
- Designed Use cases, activities, states, objects and components.
- Developed the UI pages using HTML, DHTML,Javascript, AJAX, JQUERY, JSP and tag libraries.
- Developed front-end screens using JSP and Tag Libraries.
- Performing validations between various users.
- Design ofJavaServlets and Objects using J2EE standards.
- Coded HTML, JSP and Servlets.
- Experience in writing python code and My SQL for creating generic solutions of human resource problems.
- Deployed and built the application using Maven.
- Performed testing using JUnit.
- Developed internal application using Angular andNode.JS connecting to Oracle on the backend.
- Coding xml validation and file segmentation classes for splitting large XML file into smaller segments using SAX Parser. Developed Class diagrams, Sequence diagrams using Rational Rose.
- Responsible in developing Rich Web Interface modules with Struts tags,JSP, JSTL, CSS, JavaScript, Ajax, GWT.
- Developed presentation layer using Struts framework, and performed validations using Struts Validator plug-in.
- Created SQL script for the Oracle database.
- Implemented the Business logic using Java Spring Transaction Spring AOP.
- Implemented persistence layer using Spring JDBC to store and update data in database.
- Produced web service using WSDL/SOAP standard.
- Implemented J2EE design patterns like Singleton Pattern with Factory Pattern.
- Extensively involved in the creation of the Session Beans and MDB, using EJB 3.0.
- Used Hibernate framework for Persistence layer.
- Extensively involved in writing Stored Procedures for data retrieval and data storage and updates in Oracle database using Hibernate.
- Used JIRA to track bugs.
- Extensively used Log4j for logging throughout the application.
- Produced a Web service using REST with Jersey implementation for providing customer information.
- Created new connections through application coding for better access to DB2 database and involved in writing SQL & PL SQL - Stored procedures, functions, sequences, triggers, cursors, object types.
- Implemented application using Struts MVC framework for maintainability.
- Involved in testing and deploying in the development server.
- Prepared design document forjavacomponents.
- Wrote oracle stored procedures (PL/SQL) and calling it using JDBC.
- Involved in the design of tables of the database in Oracle.
Environment: Java, Python, J2ee, SQL, REST, JIRA, Apache Tomcat, JSP, JSTL, CSS, GWT, CVS, Servlets, Struts, DB2, PL/SQL, Oracle JDBC,MVC, HTML, DHTML,Javascript, AJAX, JQUERY, JUnit.
Confidential
Java/J2EE Developer
Responsibilities:
- Created Use case, Sequence diagrams, functional specifications and User Interface diagrams using Star UML.
- Involved in development of Java, JSP, Servlets components development and deployment.
- Involved in the complete development of forms in terms of templates, pages & components.
- Developed user interfaces using JSP, HTML, XML and JavaScript.
- Involved in front end validations using JavaScript.
- Generated XML Schemas and used XML Beans to parse XML files.
- Created Stored Procedures & Functions. Used JDBC to process database calls for DB2/AS400 and SQL Server databases.
- Developed the code which will create XML files and Flat files with the data retrieved from Databases and XML files.
- Fixing post launch bugs.
- Involved in testing the entire flow of the project and providing the unit test cases for the project.
Environment: Java 1.3, Servlets, JSPs, Java Mail API, Java Script, HTML, MySQL 2.1, Swing, Java Web Server 2.0, JBoss 2.0, RMI, Rational Rose, Red Hat Linux 7.1.
