Hadoop Developer Resume
San Bruno, CA
SUMMARY
- Over 8+ years of professional IT experience with strong emphasis on Bigdata, Hadoop ecosystem related technologies in multiple industries such as Financial, Mortgage, Banking, Insurance,Healthcare and Public Sectors.
- Excellent understanding / knowledge of Hadoop architecture and various components such asBig Data and Hadoop File SystemHDFS, Job Tracker, Task Tracker, NameNode, Data Nodeand Hadoop MapReduce programming paradigm.
- Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop MapReduce, HDFS, HBase, Oozie, Hive, Sqoop, Pig, Zookeeper, Flume and Avro.
- Good Exposure on Apache Hadoop Map Reduce programming, PIG Scripting and Distributed Application and HDFS.
- Good Knowledge on Hadoop Cluster architecture and monitoring the cluster and SOLR/Lucene.
- Expertise in job workflow scheduling and monitoring tools like Oozie and Zookeeper.
- Extended Hive and Pig core functionality by writing custom UDFs.
- Experience in managing and reviewing Hadoop log files.
- Worked on NoSQL database like HBase, MongoDB and Cassandra.
- Implemented in setting up standards and processes for Hadoop based application, design, and implementation.
- Experience in using Apache Flume for collecting, aggregating and moving large amounts of data from application servers.
- Expertise in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice - versa.
- Experience in Object Oriented Analysis, Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns and Core Java design patterns.
- Experience in Amazon AWS cloud services (EC2, EBS, S3).
- Proficient in managing Hadoop clusters using Cloudera Manager Tool.
- In-depth understanding of Data Structure and Algorithms.
- Very good experience in complete project life cycle (design, development, testing and implementation) of Client Server and Web applications.
- Extensive experience working in Oracle, DB2, SQL Server,PL/SQLand My SQL database.
- Hands on experience in application development using Java, RDBMS, and Unix shell scripting.
- Familiar with Java virtual machine (JVM) and multi-threaded processing.
- Experience with web-based UI development using JQuery, UI, CSS, HTML, HTML5, XHTML and JavaScript.
- Extensive experience in middle-tier development using J2EE technologies like JDBC, JNDI, JSP, Servlets, JSP, JSF, Struts, Spring, Hibernate, JDBC, EJB, JBoss and Ajax.
- Good understanding of application servers WebLogic, WebSphere and XML methodologies (XML,XSL,XSD) including Web Services and SOAP
- Ability to adapt to evolving technology, strong sense of responsibility and accomplishment.
- Excellent analytical, problem solving, communication and interpersonal skills, with ability to interact with individuals at all levels.
TECHNICAL SKILLS
Bigdata Ecosystem: Hadoop, Map Reduce,HDFS, Hive, Pig, HBase, Zookeeper, Sqoop, Oozie
Java / J2EE Technologies: Java 6.0, J2EE,Servlets,JSP,JDBC,XML,AJAX,SOAP, WSDL
SDLC Methodologies: Agile, UML, Design Patterns (Core Java and J2EE)
Enterprise Frameworks: Ajax,MVC, Struts 2/1, Hibernate 3, Spring 3/2.5/2
Programming Languages: Java, XML, Unix Shell scripting, SQL and PLSQL
Web Technologies: HTML, DHTML, XML, XSLT, JavaScript, CSS
Web Services: Web Logic, Web Sphere, Apache Tomcat, JBoss
Databases: Oracle 11g/10g, DB2, MS - SQL Server, MySQL, MS - Access
Operating Systems: Windows 9x/NT/XP, UNIX, Linux
PROFESSIONAL EXPERIENCE
Confidential, San Bruno, CA
Hadoop Developer
Responsibilities:
- Worked with business partners to gather business requirements.
- Developed the application by using the Spring MVC framework.
- Created connection through JDBC and used JDBC statements to call stored procedures.
- Responsible for building scalable distributed data solutions using Hadoop.
- Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
- Developed the Pig UDF’S to pre-process the data for analysis.
- Implemented multiple Map Reduce Jobs in java for data cleansing and pre-processing.
- Experienced in loading data from UNIX file system to HDFS.
- Developed job workflow in Oozieto automate the tasks of loading the data into HDFS.
- Responsible for creating Hive tables, loading data and writing Hive queries.
- Effectively involved in creating the partitioned tables in Hive.
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and extracted data from Teradata into HDFS using Sqoop.
- Worked extensively with Sqoop for importing metadata from Oracle.
- Configured Sqoop and developed scripts to extract data from SQL Server into HDFS.
- Expertise in exporting analyzed data to relational databases using Sqoop.
- Implemented Fair schedulers on the Job tracker to share the resources of the Cluster for the Map Reduce jobs given by the users.
- Cluster co-ordination services through ZooKeeper.
- Responsible for running Hadoop streaming jobs to process terabytes of xmldata.
- Gained experience in managing and reviewing Hadoop log files.
Environment: Hadoop 1x, HDFS, Map Reduce, Hive 10.0, Pig, Sqoop, Hbase, Shell Scripting, Oozie, Oracle 10g, SQL Server 2008, Ubuntu 13.04, Spring MVC, J2EE, Java 6.0, JDBC, Apache Tomcat
Confidential, Sanjose CA
Hadoop Developer
Responsibilities:
- Developed the application using Struts Framework that leverages classical Model View Controller (MVC) architecture.
- Extensively worked on User Interface for few modules using JSPs, JavaScript and Ajax
- Created Business Logic using Servlets, POJO’s and deployed them on Web logic server
- Installed and Configured Apache Hadoop clusters for application development and Hadoop tools like Hive, Pig, HBase, Zookeeper and Sqoop.
- Responsible for Cluster maintenance, adding and removingcluster nodes, Cluster Monitoring and Troubleshooting, manage and review data backups and log files.
- Worked with systems engineering team to plan and deploy new Hadoop environments and expand existing Hadoop clusters.
- Monitored multiple Hadoop clusters environments using Ganglia.
- Managing and scheduling Jobs on a Hadoop cluster.
- Involved in defining job flows, managing and reviewing log files.
- Monitored workload, job performance and capacity planning using Cloud era Manager.
- Installed Oozie workflow engine to run multiple Map Reduce, Hive and Pig jobs.
- ImplementedMap Reduce programs on log data to transform into structured way to find user information.
- Responsible for loading and transforming large sets of structured, semi structured and unstructured data.
- Collected the log data from web servers and integrated into HDFS using Flume.
- Responsible to manage data coming from different sources.
- Extracted files from CouchDB and placed intoHDFSusing Sqoopand pre process the data for analysis.
- Gained experience with NOSQL database.
- Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts.
Environment: JDK 1.5, J2EE 1.4, Struts 1.3, JSP, Servlets 2.5, WebSphere 6.1, HTML, XML, JavaScript,Hadoop, HDFS, Pig, Hive, MapReduce, HBase, Sqoop, Oozie,Ganglia and Flume
Confidential, phoenix,AZ
Java/J2EE/Hadoop Developer
Responsibilities:
- Developed the application using Struts Framework that leverages classical Model View Layer (MVC) architecture UML diagrams like use cases, class diagrams, interaction diagrams, and activity diagrams were used.
- Participated in requirement gathering and converting the requirements into technical specifications.
- Extensively worked on User Interface for few modules using JSPs, JavaScript and Ajax.
- Created Business Logic using Servlets, Session beans and deployed them on Web logic server.
- Wrote complex SQL queries and stored procedures.
- Developed the XML Schema and Web services for the data maintenance and structures.
- Implemented the Web Service client for the login authentication, credit reports and applicant information using Apache Axis 2Web Service.
- Involved in loading data from UNIX file system to HDFS.
- Installed and configured Hive and also written Hive UDFs.
- Responsible to manage data coming from different sources.
- Designed the logical and physical data model, generated DDL scripts, and wrote DML scripts for Oracle 9i database.
- Used Hibernate ORM framework with Spring framework for data persistence and transaction management.
- Used struts validation framework for form level validation.
- Wrote test cases in JUnit for unit testing of classes.
- Involved in templates and screens in HTML and JavaScript.
- Involved in integrating Web Services using WSDL and UDDI.
- Provided Technical support for production environments resolving the issues, analyzing the defects, providing and implementing the solution defects.
- Built and deployed Java applications into multiple Unix based environments and produced both unit and functional test results along with release notes.
Environment: JDK 1.5, J2EE 1.4, Struts 1.3, JSP, Servlets 2.5, WebSphere 6.1, HTML, XML, ANT 1.6, JavaScript, JUnit 3.8, Oracle 9i, Shell Scripting, Hive, HDFS
Confidential, Houston, TX
Java Developer
Responsibilities:
- Used Agile Software Development for developing the application and developed Use case diagrams, Object diagrams, Class diagrams, and Sequence diagrams using UML.
- Involved in development of User Interface using JSP, HTML, Ajax and JSTL.
- Used spring‘s MVC framework at the front end of the application.
- Implemented the business logic in stateless session Beans.
- Configured and maintained the Spring Application Framework’sIoC container.
- Involved in development of DAO layer for using Hibernate.
- Involved in development of Procedures, functions and triggers in the database.
- Implemented various J2EE design patterns such as Session Façade, Business Delegate, Value Object, Data Access Object, etc.
- Used Selenium IDE for testing the web application from Mozilla Firefox.
- Used Selenium Remote Control to execute the tests from eclipse IDE.
- Developed selenium classes using selenium RC Framework.
- Used Regex Patterns to test the text content on a webpage using Selenium.
- Used Selenium Remote Control Server to act as a gateway between the AUT and Selenium Test Class.
- Used Selenium RC to work in the https secure environment by setting the run mode.
- Debugged errors occurring while working with Selenium.
- Used Assertions for writing the assert statements in the JUnittest cases.
- Used Test Driven Development in writing the test cases for developing the Java modules.
- Used JSF standard validations&JavaScript for client-side validations.
- Used hibernate for Data persistence in back end
- Packaged and deployed the application in Web sphere server in different environments.
- Fine tuned the application by using JProbe and load runner tools.
- Developed unit test cases using JUnit and Mock Objects.
- Used CVS as version control system for the source code and project documents.
- Actively involved throughout the life cycle of the project from requirements gathering to maintenance and optimizing the system.
Environment: RSA 6.0, Web sphere, Selenium, JSP, EJB, JNDI, JSF, Java Script, Ajax, Perl, XSLT, XQuery, UML, SVN, ANT, JUnit, Spring, XML,UNIX, Hibernate, DB2, and Toad
Confidential, Chicago ILJava Developer
Responsibilities:
- Design and developed Web services based on SOAP, WSDL, JAXWSusing Spring.
- Involved in designing the xml schema for using web services.
- Designed and developed Data layer for Client communicating both Oracle and Sybase at any time.
- Designed class diagrams and sequence diagrams using Microsoft Visio 2007.
- Migrated complex queries and stored procedures from Sybase to Oracle.
- Developed tools to generate automated send views and ret views to serialize the data to the mainframe.
- Developed and deployed Message Driven Beans to apply same adjustment for multiple airbills asynchronously.
- Used multithreading on the client to process huge requests.
- Created different state machines to accomplish the dependant tasks individually one after another.
- Used ASN encoding to send the data across the network and used MIIG API to talk to mainframe server.
- Migrated functionalities developed in C(procedure Oriented Language) code to JAVA without missing business rules.
- Created JSP page to modify loglevels dynamically without restarting the server.
- Followed Scrum methodology to keep track of all requirements.
- Involved in creating automated builds using ANT for the client and Maven to build/deploy onto weblogic server.
Environment: Java, J2EE, SOAP, XML, XSD, WSDL, JAXB, SOAP UI, MDB, ANT/Maven, JUnit, SVN, Eclipse, Oracle, PL/SQL, Weblogic
Confidential
Java/J2EE Interface Developer
Responsibilities:
- Created Use case, Sequence diagrams, functional specifications and User Interface diagrams using Star UML.
- Involved in complete requirement analysis, design, coding and testing phases of the project.
- Participated in JAD meetings to gather the requirements and understand the End Users System.
- Developed user interfaces using JSP, HTML, XML and JavaScript.
- Generated XML Schemas and used XML Beans to parse XML files.
- Created Stored Procedures & Functions. Used JDBC to process database calls for DB2/AS400 and SQL Server databases.
- Developed the code which will create XML files and Flat files with the data retrieved from Databases and XML files.
- Created Data sources and Helper classes which will be utilized by all the interfaces to access the data and manipulate the data.
- Developed web application called iHUB (integration hub) to initiate all the interface processes using Struts Framework, JSP and HTML.
- Developed the interfaces using Eclipse 3.1.1 and JBoss 4.1. Involved in integrated testing, Bug fixing and in Production Support.
Environment: Java 1.3, Servlets, JSPs, Java Mail API, Java Script, HTML, MySQL 2.1, Swing, Java Web Server 2.0, JBoss 2.0, RMI, Rational Rose, Red Hat Linux 7.1.