Hadoop Developer Resume
Camphill, PA
SUMMARY
- 8+ years of overall experience in applications development, integration, maintenance and support using Java/J2EE, Hadoop platforms.
- 3+ years of experience in Hadoop platform (Apache, Cloudera and Hortonworks).
- Expertise in understanding, solving big data problems and plan, design, create a Hadoopcluster, setup and configure commonly used Hadoop ecosystem components such as HDFS, Map Reduce, Hive, Pig, HBase, Sqoopand Oozie.
- Strong knowledge in development of Object Oriented and Distributed applications.
- Written unit test cases using JUnit and MRUnit for MapReduce jobs.
- Good knowledge in programming JDBC, Servlets and JSP.
- Experience in working with ETL workflow.
- Developed Map Reduce programs to perform analysis.
- Have hands on experience on various DB platforms like Oracle, MySQL, DB2 and MS SQL Server.
- Expertise in writing automation test cases using Selenium framework.
- Good knowledge in NoSQL databases including Cassandra and MongoDB.
- Experienced with Agile SCRUM methodology, involved in design discussions and work estimations, takes initiatives, very proactive in solving problems and providing solutions.
- Hands on experience in Analysis, Design, Coding and Testing phases of Software Development Life Cycle (SDLC).
- Provided technology, domain s through regular knowledge sharing sessions, guided team members through hands - on lab sessions which enabled them quickly learn the essentials.
- Have strong interpersonal skills resulting in exceptional rapport with people, ability to meet deadlines, adaptive in learning and working on various technologies.
TECHNICAL SKILLS
Languages: Java, Java Script, XML, XSL, XSLT, SQL and PL/SQL
Web Frameworks: Servlets, JSP, Spring MVC, Struts, Hibernate, Apache Axis2
Servers: Confidential WebSphere Application Server, JBoss&Apache Tomcat
IDE: Confidential Rational Application Developer (RAD), Eclipse, XCode
Databases: Confidential DB2, Oracle, My SQL & Microsoft SQL Server
NoSQL Databases: Cassandra, MongoDB
BigData: Hadoop HDFS, MapReduce, HBase, Hive, Sqoop
Hadoop Distributions: Apache, Cloudera, Hortonworks
Operating Systems: Cent OS, MacOS X, RHEL, Ubuntu, Windows 7
Version Control: Confidential Rational ClearCase, Microsoft Visual Source Safe (VSS), Microsoft Team Foundation Server (TFS)
Testing Frameworks: Selenium WebDriver, Apache JUnit, Apache MRUnit
PROFESSIONAL EXPERIENCE
Confidential, CampHill, PA
Hadoop Developer
Responsibilities:
- Installed and configured Cloudera Manager for easy management of existing Hadoop cluster.
- Deployed Network File System for Namenode metadata backup.
- Worked on setting up high availability for major production cluster and designed automatic failover control using zookeeper and quorum journal nodes.
- Used Sqoop to transfer data between RDBMS and HDFS.
- Involved in collecting and aggregating large amounts of streaming data into HDFS using Flume and defined channel selectors to multiplex data into different sinks.
- Implemented complex map reduce programs to perform joins on the Map side using distributed cache.
- Designed and implemented custom writable, custom input formats, custom partitions and custom comparators in Mapreduce.
- Thoroughly tested Mapreduce programs using MRUnit and Junit testing frameworks.
- Responsible for troubleshooting issues in the execution of Mapreduce jobs by inspecting and reviewing log files.
- Converted existing SQL queries into Hive QL queries.
- Implemented UDFs, UDAFs, UDTFs in java for hive to process the data that can’t be performed using Hive inbuilt functions.
- Wrote Pig Latin scripts for advanced analytics on data for recommendations.
- Effectively used Oozie to develop automatic workflows of Sqoop, Mapreduce and Hive jobs.
- Organized daily SCRUM meeting with team, prioritize product backlog items and responsible for timely delivery and deployment of product releases.
Environment: CDH4 Cloudera Distribution, Sqoop, Pig Latin, Hive, Flume, HDFS, MapReduce, Eclipse IDE, UNIX Shell Scripting
Confidential, Chicago, IL
Hadoop Developer
Responsibilities:
- Analyzed the Big Data business requirements and transformed it into Hadoop centric technologies.
- Worked with Big data Analysts, designers and scientists in troubleshooting issues with Sqoop, Hive, Pig, Flume etc.
- Worked on streaming log data into HDFS from web servers using Flume.
- Implemented custom interceptors for flume to filter data as per requirement.
- Used compression codecs to compress data in HDFS.
- Used Hive and Pig to analyze data in HDFS to identify issues and behavioral patterns.
- Configured and deployed Hive metastore using MySQL and Thrift server.
- Created internal and external Hive tables and defined static and dynamic partitions as per requirement for optimized performance.
- Wrote Pig Latin scripts for running advanced analytics on the data collected.
- Implemented Pig UDFs for evaluation, filtering, loading and storing of data for functionalities which cannot be achieved using built-in Pig functions.
- Exported the analyzed data using Sqoop to Oracle database.
- Configured daily workflow for extraction, processing and analysis of data using Oozie.
- Proactively involved in ongoing maintenance, support and improvements in Hadoop cluster.
Environment: Java 6, Eclipse,MySql, Sub Version,Hadoop,Hive,Pig, HBase, Linux,MapReduce, Hive, Java (JDK 1.6),MapReduce,Cassandra
Confidential, Jamesburg, NJ
Hadoop Developer
Responsibilities:
- Exported data from DB2 to HDFS using Sqoop and NFS mount approach.
- Moved data from HDFS to Cassandra using Map Reduce and BulkOutputFormat class.
- Developed Map Reduce programs for applying business rules on the data.
- Developed and executed hive queries for denormalizing the data.
- Works with ETL workflow, analysis of big data and loaded them into hadoop cluster.
- Installed and configured Hadoop Cluster for development and testing environment.
- Implemented Fair scheduler on the Job tracker to share the resources of the cluster for the map reduces jobs given by the users.
- Automated the work flow using shell scripts.
- Performance tuning of the hive queries, written by other developers.
Environment: Linux, Java, Map Reduce, HDFS, DB2, Cassandra, Hive, Pig, Sqoop, FTP
Confidential, Bloomington, IL
Java Developer
Responsibilities:
- Worked on front end of the application which is called CLIQ.
- Worked on Quote Summary screen which displays all the Information that has been selected throughout the application.
- Used Service-Oriented Architecture (SOA) to communicate between different applications.
- Responsible for the source code maintenance using version control using Source Jammer Versions system.
- Implemented the MVC architecture using Struts framework. Implemented J2EEdesign patterns like Service Locator, Business delegate design patterns.
- Developed the front-end components usingJava Server Pages, Servers HTML and XML.
- Coding Different Java helper and validation classes for the Application logicandutilities.
- Process data feeds from external systems and generate feeds from the application database and send to other systems.
- Performed logging of the application using log4j framework.
- Involved in development of Unit test cases and testing the application using JUnit 3.
Environment: Java, websphere6, RAD 7, Servlets 2.3, JSP, JSTL, Expression Language, Custom tags, EJB 2.0, Log4j, JavaScript, JUNIT, JMS, XML, XSL, XSLT, TML, DHTML, JDBC, PL/SQL, Web Services, AJAX, Solaris, RationalClearcase, Rational Clearquest iGATE Global Solutions, India
Senior Software Engineer
Confidential
Responsibilities:
- Developed the code using the struts framework.
- Involved in requirement analysis.
- Developed UI components using JSP and JavaScript.
- Involved in writing the technical design document.
- Developed the front end for the site based on (MVC) design pattern Using Struts framework.
- Created a data access layer to make rest of the code database independent.
- Developed JSPs, ServLets and created java beans for the application.
- Developed sample requests and responses for testing web services.
- Deployed web applications on server using Apache Tomcat.
- Developed new code for the change requests.
- Developed complex PL/SQL queries to access data.
- Coordinated across multiple development teams for quick resolution to blocking issues.
- Prioritized tasks and coordinated assignments with the team.
- Performed on call support on a weekly rotation basis.
- Performed manual and automated testing.
- Involved in writing and updating the Test cases in the Quality tool.
Environment: JSP, Java Bean, Servlets, Oracle, HTML & JAVASCRIPT, JDBC, PL/SQL, The web-tier consists of Apache
Confidential
Java Software Engineer
Responsibilities:
- Involved in Coding and Unit in Job Submission/Controller module consists of Creation, Modification and Deletion.
- Involved in coding java classes and Jsp using Struts Framework.
- Design and development of SOA, web services components using XML, SOAP and JAX-WS.
- Development of PL/SQL and Stored procedures for Oracle.
- Developed and deployment of web services.
- Web Services based on AXIS distribution for publishing and consuming internally.
- Defect mitigation or fixing issues/ bugs.
- Setting up build environment by writing Ant build.xml, taking build, configuring, and deploying of the application in all the servers (dev, stage and production servers).
Environment: Java, Struts framework, J2EE, JSP, Hibernate, SOA, webservices(SOAP), WSDL, SQLServer 2000, My Eclipse, JBoss (4.0/5.0) Appication Server
