We provide IT Staff Augmentation Services!

Hadoop Developer Resume

2.00/5 (Submit Your Rating)

Pittsburgh, PA

SUMMARY:

  • 10+ years of overall IT experience and 3+ Years of comprehensive experience as a Apache Hadoop Developer. Expertise in writing Hadoop Jobs for analyzing data using Hive, Pig and oozie.
  • Sun Certified Java Programmer with over 7+ years of Extensive programming experience in developing web based applications and Client - Server technologies using Java, J2EE.
  • Good knowledge of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce concepts.
  • Experience in working with MapReduce programs using Hadoop for working with Big Data.
  • Experience in analyzing data using Hive QL, Pig Latin and custom MapReduce programs in Java.
  • Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
  • Working experience on designing and implementing complete end-to-end Hadoop Infrastructure including PIG, HIVE, Sqoop, Oozie, Flume and zookeeper.
  • Experience in providing support to data analyst in running Pig and Hive queries.
  • Developed Map Reduce programs to perform analysis.
  • Performed Importing and exporting data into HDFS and Hive using Sqoop.
  • Experience in writing shell scripts to dump the Sharded data from MySQL servers to HDFS.
  • Experience in designing both time driven and data driven automated workflows using Oozie.
  • Experience in setting up Infiniband network and build Hadoop cluster to improve the map reduce performance.
  • Experience in performance tuning the Hadoop cluster by gathering and analyzing the existing infrastructure.
  • Experience in automating the Hadoop Installation, configuration and maintaining the cluster by using the tools like puppet.
  • Working Experience in building infrastructure from bare-metal by utilizing DHCP, PXE, DNS, KICKSTART, and NFS.
  • Experience in setting up monitoring infrastructure for Hadoop cluster using Nagios and Ganglia.
  • Experience in working with flume to load the log data from multiple sources directly into HDFS.
  • Strong debugging and problem solving skills with excellent understanding of system development methodologies, techniques and tools.
  • Worked in complete Software Development Life Cycle (analysis, design, development, testing, implementation and support) in different application domain involving different technologies varying from object oriented technology to Internet programming on Windows NT, Linux and UNIX/ Solaris platforms and RUP methodologies.
  • Familiar with RDBMS concepts and worked on Oracle 8i/9i, SQLServer 7.0., DB2 8.x/7.x
  • Involved in writing shell scripts, Ant scripts for Unix OS for application deployments to production region.
  • Exceptional ability to quickly master new concepts and capable of working in-group as well as independently with excellent communication skills.

TECHNICAL SKILLS:

Languages/Tools: Java, C, C++, VB, XML, HTML/XHTML, HDML, DHTML.

Big Data: Hadoop, Map Reduce, Hive, Pig, Sqoop and MRUnit

J2EE Standards: JDBC, JNDI, JMS, Java Mail & XML Deployment Descriptors.

Web/Distributed: J2EE, Servlets 2.1/2.2, JSP 2.0, Struts 1.1, Hibernate 3.0, JSFTechnologies JSTL1.1,EJB 1.1 /2.0, RMI,JNI, XML,JAXP,XSL,XSLT, UML, MVCSTRUTS,Spring 2.0, Corba, Java Threads.

Operating System: Windows 95/98/NT/2000/XP, MS-DOS, UNIX, Linux6.2

Databases: Oracle 8i/9i, MS SQL Server 2000, DB2, MS Access & MySql.

Browser Languages: HTML, XHTML, CSS, XML, XSL, XSD, XSLT.

Browser Scripting: Java script, HTML DOM, DHTML, AJAX.

App/Web Servers: IBM Websphere 5.1.2/5.0/4.0/3.5 , BEA Weblogic 5.1/7.0, JdeveloperApache Tomcat, JBoss.

GUI Environment: Swing, AWT.

Messaging &: SOAP, WSDL,UDDI, XML, SOA, JAX-RPC, IBM WebSphere MQ v5.3

Web Services Technology: JMS.

Networking Protocols: HTTP, HTTPS, FTP, UDP, TCP/IP, SNMP,SMTP,POP3.

Testing &Case Tools: JUnit, Log4j, Rational Clear case, CVS, ANT, JBuilder.

PROFESSIONAL EXPERIENCE:

Confidential, Pittsburgh, PA

Hadoop Developer

Responsibilities:

  • Installed and configured Hadoop Mapreduce, HDFS, Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
  • Installed and configured Pig and also written PigLatin scripts.
  • Wrote MapReduce job using Pig Latin.
  • Have solid understanding of REST architecture style and its application to well performing web sites for global usage.
  • Involved in ETL, Data Integration and Migration. Imported data using Sqoop to load data from Oracle to HDFS on regular basis.
  • Developing Scripts and Batch Job to schedule various Hadoop Program.
  • Written Hive queries for data analysis to meet the business requirements.
  • Creating Hive tables and working on them using Hive QL. Importing and exporting data into HDFS from Oracle Database and vice versa using Sqoop.
  • Implemented test scripts to support test driven development and continuous integration.
  • Responsible to manage data coming from different sources.
  • Load and transform large sets of structured, semi structured and unstructured data.
  • Experience in managing and reviewing Hadoop log files.
  • Worked on Hive for exposing data for further analysis and for generating transforming files from different analytical formats to text files.
  • Managing and scheduling Jobs on a Hadoop cluster.
  • Installed and configured Hadoop MapReduce, HDFS, Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
  • Importing and exporting data into HDFS and Hive using Sqoop.
  • Involved in creating Hive tables, loading with data and writing hive queries, which will run internally in map, reduce way.
  • Used Pig as ETL tool to do transformations, event joins, filter bot traffic and some pre-aggregations before storing the data onto HDFS.
  • Written Hive queries for data analysis to meet the business requirements.
  • Involved in writing Hive scripts to extract, transform and load the data into Database.
  • Used JIRA for bug tracking.
  • Used CVS for version control.

Environment: Hadoop, Hive, Linux, MapReduce, HDFS, Hive, Pig, Sqoop, Shell Scripting, Java (JDK 1.6), Java 6, Eclipse, Oracle 10g, PL/SQL, SQL*PLUS, Toad 9.6, Linux, JIRA 5.1, CVS, JIRA 5.2.

Confidential, Pittsburgh

Hadoop Developer

Responsibilities:

  • Developed Oozie Workflows for daily incremental loads, which gets data from Teradata and then imported into hive tables.
  • Developed pig scripts to transform the data into structured format and it are automated through Oozie coordinators.
  • Developed Hive queries for Analysis across different banners.
  • Loading the data from the different Data sources like (Teradata and DB2) into HDFS using sqoop and load into Hive tables, which are partitioned.
  • Developed Hive UDF’s to bring all the customers email id into a structured format.
  • Developed bash scripts to bring the Tlog files from ftp server and then processing it to load into hive tables.
  • All the bash scripts are scheduled using Resource Manager Scheduler.
  • Moved data from HDFS to Cassandra using Map Reduce and BulkOutputFormat class.
  • Developed Map Reduce programs for applying business rules on the data.
  • Developed and executed hive queries for denormalizing the data.
  • Supported Data Analysts in running Map Reduce Programs.
  • Worked on importing and exporting data into HDFS and Hive using Sqoop.
  • Worked on analyzing data with Hive and Pig.
  • Experience in Implementing Rack Topology scripts to the Hadoop Cluster.
  • Manage the day-to-day operations of the cluster for backup and support.

Environment: Hadoop, Hive, Linux, MapReduce, HDFS, Hive, Pig, Sqoop, Java 1.5, J2EE, Struts 2.0, WebSphere 7.0, IBM RAD 8.0, Rational Clearcase 7.0, XML, JAXP, Castor, XSL, XSLT, XML Schema(XSD), WSDL 2.0, SOAP,JSP 2.2, CSS, Servlets, JavaScript, HTML, JMS, AXIS 2, Open source technologies (ANT, LOG4j and Junit), Oracle 10g, UNIX.

Confidential, Silver Spring

Java/J2EE Developer

Responsibilities:

  • Involved in design, development and maintenance of all Confidential web applications.
  • Designed and developed Playcount Service for all web applications videos using JMS to audit the video view count in data base & display the same count in UI screen.
  • Extensively used IBM Rational Application Developer 7.5 for building, testing, and deploying applications.
  • Worked on VAU (Video Automation Utility) application to get the changed and new videos from BrightCove and process and store these videos in their local Database.
  • Designed and developed SMILService in Spring MVC framework to play all discovery videos depends on the player request.

Environment: Java 1.6, J2EE, Spring MVC 2.5, WebSphere 6.1, IBM RAD 7.5, XML, XML Spy, XSD, XSL, XSLT, JSP, Toplink, JavaScript, JTEST, HTML, Json, Velocity, Agile, Scrum, CMS, Open source technologies (ANT, LOG4j and Junit), Oracle 10g, Toad, UNIX.

Confidential, Chicago

Java/J2EE Developer

Responsibilities:

  • Worked with MVC Bridje framework ( Confidential proprietary framework that is built with Confidential standard J2EE technologies) at the front end and uses Toplink framework to connect to Oracle database at the backend.
  • Implemented business logic is present at the middle layer which is written in Java & J2EE. Also the business rules are defined in xml which will be parsed by the Java rules engine for risk calculation.
  • Confidential interacts with the other systems worked with SOAP (Simple Object Access Protocol) and JMS.
  • Confidential is developed using Agile programming methodology, we use the following automated tools for improving quality and performance of the application.Used Junits for unit testing and it is automated and run as part of the build.Used Selenium for front end functional testing.

Environment: Java 1.5, J2EE, MVC Bridje framework, WebSphere 6.1, IBM RAD 7.5, XML, XML Spy, XSD, XSL, XSLT, JSP, Toplink, JavaScript, JTEST, HTML, Vitria, Open source technologies (ANT, LOG4j and Junit), Oracle 10g, Toad, UNIX.

Confidential, Pittsburgh

Java/J2EE Developer

Responsibilities:

  • Involved in Business requirements gathering, Design, Development and unit testing of BillPay Account Accelerator (BPAA) & Alphanumeric ID prjects.
  • Involved in maintenance & development of Confidential .com and their related web sites like pncvirtualwallet , Wealth Management & Mutual Funds etc.
  • Responsible for developing use cases, class and sequence diagram for the modules using UML and Rational Rose .
  • Involved in preparation of docs like Functional Specification document and Deployment Instruction documents.
  • Set up the deployment environment on WebSphere 6.1 Developed system preferences UI screens using JSP2.0 and HTML.
  • Used Java Script for Client side validations.
  • Code and Unit Test according to client standards. Provide production support and quickly resolving the issues until Integration Test is passed.
  • Fix defects as needed during the QA phase, support QA testing, troubleshoot defects and identify the source of defects.
  • Used JMS for Point-to-Point asynchronous messaging for high transactional Banking operation.
  • Involved in preparation of unit and system test cases and testing of the module in 3 phases named unit testing and system testing and regression testing.
  • Involved in writing shell scripts, Ant scripts for Unix OS for application deployments on production region.
  • Developed core banking business components as a Web Service for enterprise-wide SOA Architecture strategy.
  • Used Rational ClearCase as source control management system.
  • Implemented SOA architecture with web services using SOAP, WSDL, UDDI and XML.
  • Involved in deployments in all environments like Dev, Test,UAT and prod respctively.
  • Involved in design Credit Card Service layer on mainframe with MQ series and WBI. Provide XML based messaging service to front-end applications.
  • Extensively used IBM RAD 7.1 IDE for building, testing, and deploying applications.
  • Responsible for developing use cases, class and sequence diagram for the modules using UML and Rational Rose 2000
  • Worked with Single Sign-On (SSO) using SAML for retrieving data from third party applications like Yodlee.

Environment: Java (jdk1.5), J2EE, WebSphere 6.1, IBM RAD 7.5, Rational Clearcase 7.0, XML, JAXP, XSL, XSLT, XML Schema(XSD), WSDL 2.0, SAML 2.0, AJAX 1.0, Web Services, SOA, JSP 2.2, CSS, Servlets, JProfiler, Struts 2.0, Spring, Rational HATS, JavaScript, JCF, HTML,IBM DB2, JMS, AXIS 2, Swing, MQ, Open source technologies (ANT, LOG4j and Junit), Oracle 10g, UNIX

Confidential, Chicago

Java/J2EE Developer

Responsibilities:

  • Responsible for developing use cases, class and sequence diagram for the modules using UML and Rational Rose
  • Extensively used IBM Rational Application Developerr 7.0 for building, testing, and deploying applications.
  • Developed the Java code for validating the data according to business rules and used Castor to convert data into array of xml strings.
  • Worked with BPM workflow using Vitria.
  • Used JDBC 3.0 for Oracle database connection and written number of stored procedures for retrieveing the data and validated that data against the business rules.
  • Involved in Unit & Integration Testing for all Facultative,Treaty & Wire Modules.
  • Developed Detailed Design document for all three modules.

Environment: Java 1.4.2, J2EE, Struts 1.3, WebSphere 6.0, IBM RAD 7.0, XML, XML Spy, XSD, XSL, XSLT, JSP, JDBC, Jprobe, JavaScript, JTEST, HTML, Vitria, Open source technologies (ANT, LOG4j and Junit), Oracle 10g, Toad, UNIX

Confidential, NY

Java/J2EE Developer

Responsibilities:

  • Responsible for developing use cases, class and sequence diagram for the modules using UML and Rational Rose
  • Extensively used IBM Rational Application Developer for building, testing, and deploying applications.
  • Developed Swing based GUI using JDK 1.4 as per proprietary application development framework.
  • Used STRUTS Framework based on (MVC) Model View Controller in WSAD.
  • Developed the presentation layer and GUI framework in HTML, JSP and Client-Side validations were done.
  • Involved in Java code, which generated XML document, which in turn used XSLT to translate the content into HTML to present to GUI.
  • Written some stored procedures to retrieve the data from MS Sequel server 2005 database.
  • Developed Session Beans on WebLogic for the transactions in the application.
  • Utilized RAD to create JSPs/Servlets/EJBs that pulled information from a MS Sequel server database and sent to a front end GUI for end users.
  • Used A gile methodology to develop software applications.In the database end, responsibilities included creation of tables, triggers, stored procedures, sub-queries, joins, integrity constraints and views.
  • Involved in communicating with offshore team to resolve the applications production issues and to deliver the best quality application enhancemnts to the client

Environment: Java 1.4, J2EE, Struts 1.3, WebLogic, IBM RAD 6.0, EJB, XML, XML Spy, XML Schema, XSLT, XSD, JSP, JAXP, Servlets, Hibernate, JPA, iBatis, JavaScript, Perl, ANT, Message Driven Beans(MDB), LOG4j, AXIS 1.2, Jprobe, Junit, agile methodology, JSF, Faces, Swing, Web Services, SOA, HTML, Linux, MS Sequel server 2005, UNIX

Confidential, Southfield, MI

Java/J2EE Developer

Responsibilities:

  • Developed the action and action form classes, based on the Struts framework, to handle the pages.
  • Created tile definitions, struts-config files and resource bundles using Struts framework.
  • Implemented Action classes using Struts framework.
  • Implemented Struts and Spring frameworks.
  • Used design patterns like Business Delegate , Session Façade and Data Access Object.
  • Used JMS for sending the messages to the Export Queue.
  • Deployed and tested the JSP pages in WebSphere server.
  • Developed and participated Client application development using Swing/JFC components.
  • Used IBATIS for developing Object/Relational mapping.
  • Developed Microsoft based Crystal reports and integrated with Java Environment.
  • Developed the Session Beans and deployed them in WebSphere application server.

Environment: Java 1.4, J2EE, JSPs, Struts 1.1, EJB, Spring 2.0, JMS, JTEST, Jprobe, WebSphere 5.x, IBM WSAD 5.x, PL/SQL, SQL Navigator, Hibernate, iBatis, Perl, XML, XSD, LDAP, POS(point of sale),Oracle 9i, Toad, Swing, HTML, JavaScript, UML, Windows 2000

Confidential, Reston, VA

Java/J2EE Developer

Responsibilities:

  • Involved in developing business logic components using JavaBeans and Servlets.
  • Involved in utilizing WebLogic specific connection pools in order to interact with the business data from the business components.
  • Developed EJB and Web specific deployment descriptor files.
  • Designed and Developed User Interface using JSP.
  • Developed Server-side validation classes using JDBC calls.
  • Designed user interface HTML, Swing, CSS, XML, Java Script and JSP.
  • Implemented the presentation using a combination of Java Server Pages (JSP) to render the HTML and a well-defined API interface to Allow access to the application services layer.
  • Implemented Configuration of Engage Administration and property files.
  • Input validations were done using JavaScript.
  • Developed applications using Mobile Supply Chain.

Environment: (JDK 1.3), J2EE, Servlets, JSP, JSF, EJB, JMS, JavaScript, HTML, Swing, CSS, XML, XML Spy, Eclipse, Internet Explorer 5.0, Linux, Weblogic, DB2 7.x.

We'd love your feedback!