Hadoop Developer Resume
Broomfield, CO
SUMMARY:
- Over 7+ years of professional IT experience in all phases of Software Development Life Cycle which includes hands on experience in Java/J2EE technologies and Big Data Analytics.
- 3+ years of work experience in ingestion, storage, querying, processing and analysis of Big Data with hands on experience in Hadoop Ecosystem development including MapReduce, HDFS, Hive, Pig, Mahout Hbase, Zookeeper, Sqoop, Flume, Oozie, Spark and Yarn.
- Excellent understanding of Hadoop architecture and different daemons of Hadoop clusters which include Name Node and Data Node, Task Tracker, Job Tracker.
- Strong understanding and strong knowledge in NoSQL databases like HBase, MongoDB & Cassandra.
- Worked with Hive/HQL to query data from Hive tables in HDFS.
- Involved in developing Hive DDLs to create, alter and drop Hive tables.
- Importing and Exporting of data from RDBMS to HDFS and vice versa using Sqoop.
- Expertise in using Oozie for configuring job flows.
- Developed Batch Processing jobs using Java Map Reduce, Pig and Hive.
- Experience in importing streaming data into HDFS using Flume sources, and Flume sinks and transforming the data using Flume interceptors.
- Experience in cluster coordination using Zookeeper.
- Worked on writing Map Reduce programs to perform Data processing and analysis.
- Good knowledge of Data warehousing concepts and ETL processes.
- Good Knowledge and experience in Hadoop Administration.
- Enhanced and optimized product Spark code to aggregate, group and run data mining tasks using the Spark framework.
- Strong experience in Object - Oriented Design, Analysis, Development, Testing and Maintenance .
- Hands on experience on Hortonworks and Cloudera Hadoop environments.
- Developed test cases in JUnit for unit testing of MapReduce Jobs.
- Extensive experience in middle-tier development using J2EE technologies like JDBC, JNDI, JSP, Servlets, JSP, JSF, Struts, Spring, Hibernate, JDBC, EJB. Experience in client-side development with HTML, CSS, JavaScript, and JQuery.
- Experienced in deploying, configuring, maintaining applications on application servers such as Jboss, Web sphere (8.5, 8.0), Web logic, JRUN4, GlassFish and Tomcat.
- Experience with log tools: Maven, JUnit, Log4j and Ant.
- Experience in component design using UML Design-Use Case, Class, Sequence, and Development, Component diagrams for the requirements
- Good experience in implementation in version control like Rational Clear case, TFS, CM synergy, CVS.
- Have excellent analytical, problem solving, communication and interpersonal skills, with ability to interact with individuals at all levels and can work as a part of a team as well as independently and quick learning skills with proven ability to adapt to different project environments.
TECHNICAL SKILLS:
Hadoop Eco System: Hadoop 1.x/2.x(Yarn), HDFS, Map Reduce, Mongo, HBase, Hive, PIG, Zookeeper, Sqoop, Oozie, Flume, Storm, Eclipse, Cloudera-desktop and SVN
Java/J2EE Technologies/Frameworks: JDBC, Servlets, JSP, RMI, JNDI, EJB, XML, Custom Tags, JSTL, JPA. AJAX, Struts 1&2, Spring Framework, Hibernate, AngularJS.
Web Technologies: OOPs, Data Structure, Exception Handling, multi-Threading, HTML5,XHTML,DHTML,CSS3,XML,JSF,JavaScript,JSON,jQuery,REST, Soap Web Service, Bootstrap.
Databases: Oracle, MySQL, Sybase, PostgreSQL, Derby, SQL,IBM DB2 9.x.
Web / Application Servers: Apache Tomcat, JBoss, WebSphere, WebLogic
No SQL Databases: HBase, MongoDB & Cassandra.
Build Tool: Maven, Ant
Development Tools: Eclipse, RAD/RSA (Rational Software Architect), IBM DB2 Command Editor, SQL Developer, Microsoft Suite (Word, Excel, PowerPoint, Access), Open Office Suite (Editor, Calc etc..),VM Ware
Operating Systems: UNIX, Window XP, Mac OS.
Source Control: CVS, SVN, Perforce
PROFESSIONAL EXPERIENCE:
HADOOP DEVELOPER
Confidential, Broomfield, CO
RESPONSIBILITIES:
- Collecting the business requirements from business partners/stakeholders.
- Worked extensively for importing the data using Sqoop from PRPD and KPDB databases.
- Collecting the data from various data sources like ORACLE/SOAP/REST/FTP calls from external systems and importing data into HDFS .
- Wrote MapReduce Programs in Java for generating complex reports for business users.
- Created Hive tables for storing usage information.
- Wrote Hive and Pig queries to aggregate the data and send to various systems.
- Wrote Hive/Pig UDF s to pre-process the data for analysis.
- Used Talend for ETL and building Big Data components.
- Helping the business analysts to write complex Hive/Pig queries.
- Designed and created HBase Tables for storing MCP data.
- Experience in predictive analytics on top of the customer usage data using Spark/R language.
- Compared read/wirte performances between HBase / Cassandra .
- Real time data streaming using Spark/Storm/Cassandra/LinkedIn Databus.
- Developed KMeans programs using Spark MLib to analyze the usage data for the customers.
- Installation and configuration of Hadoop components in Amazon Could (AWS).
- Involved in troubleshooting the production issues and providing production support.
- Streamlined the Teradata scripts and shell scripts migration process on the UNIX box.
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports. Developed Hive queries for the analysts.
- Involved in analysis of end user requirements and business rules based on given documentation and working closely with tech leads and analysts in understanding the current system.
- Collected statistics every week on the tables to improve performance.
- Developed unit test plans and involved in system testing .
ENVIRONMENT: Hadoop (HDFS, Map Reduce, Hive, Pig, Spark,HBase, Flume, Sqoop, R, Mongo DB), Core Java, Teradata.
HADOOP DEVELOPER
Confidential, Atlanta,GA
RESPONSIBILITIES:
- Responsible for building scalable distributed data solutions using Hadoo p.
- Wrote multiple MapReduce programs in Java for Data Analysis
- Wrote MapReduce job using Pig Latin and Java API.
- Performed performance tuning and troubleshooting of MapReduce jobs by analyzing and reviewing Hadoop log files
- Developed pig scripts for analyzing large data sets in the HDFS .
- Collected the logs from the physical machines and the OpenStack controller and integrated into HDFS using Flume.
- Knowledge on handling Hive queries using Spark SQL that integrates with Spark environment.
- Responsible for creating Hive tables , loading the structured data resulted from MapReduce jobs into the tables and writing hive queries to further analyze the logs to identify issues and behavioral patterns.
- Worked on Sequence files, RC files, Map side joins, bucketing, partitioning for Hive performance enhancement and storage improvement.
- Sqoop jobs, PIG and Hive scripts were created for data ingestion from relational databases to compare with historical data.
- Utilized Storm for processing large volume of datasets.
- Used Kafka to load data in to HDFS and move data into NoSQL databases viz. Cassandra
- Used OOZIE Operational Services for batch processing and scheduling workflows dynamically.
- Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from UNIX, NoSQL and a variety of portfolios.
- Involved in submitting and tracking Map Reduce jobs using Job Tracker.
- Involved in creating Oozie workflow and Coordinator jobs to kick off the jobs on time for data availability.
- Used Visualization tools such as Powerview for excel, Tableau for visualizing and generating reports.
- Implemented Hive Generic UDF' s to implement business logic.
- Involved in story-driven agile development methodology and actively participated in daily scrum meetings.
ENVIRONMENT: Hadoop, Map Reduce, HDFS, Pig, Hive, Sqoop, Flume, Oozie, Java, Unix/Linux, Teradata, Zookeeper, Tableau, HBase, Cassandra, Kafka
SR. JAVA/J2EE DEVELOPER
Confidential, Rochester, NY
RESPONSIBILITIES:
- Involved in various phases of Software Development Life Cycle (SDLC) of the application like Requirement gathering, Design, Analysis and Code development.
- Developed a prototype of the application and demonstrated to business users to verify the application functionality.
- Attended everyday Enterprise Workflow Stand-Up meeting to provide current status, thus implementing Agile (Scrum) Methodology.
- Designed Use case diagrams, Class diagrams and Sequence diagrams using UML.
- Developed and implemented the DAO design pattern including JSP, Servlets, and Form Beans and DAO classes.
- Used spring validation framework for performing data validation.
- Implemented dynamic web interface using AJAX.
- Developed JSP fragments for control of the business processes in the middle-tier and did its integration.
- Configured and Used BEA Weblogic Server for deployment of the application.
- Developing Ant and Maven scripts to automate the compilation, deployment and testing of Web and J2EE applications to the above platforms
- Implemented complex back-end component to get the count in no time against large size MySQL database (about 4 crore rows) using Java multi-threading .
- Used JSON for storing and exchanging test information, as it is smaller and easier to parse compared to XML.
- Use Eclipse as IDE tool to develop the application and JIRA for bug and issue tracking
- Supported Oracle EBS units and resolved technical programming problems.
- Developing Ant, Maven and Shell scripts to automatically compile, package, deploy and test J2EE applications to a variety of WebSphere platforms - BPM, MQ, ESB, Portal etc
- Performed Unit Testing on the application using Junit.
- Created and deployed web pages using HTML5, CSS, JSP, JavaScript and jQuery.
- Integrated usage of stored procedures, stored functions and cursors using Oracle PL and SQL.
- Used IBM Rational Application Developer 8.0 with WebSphere Application Server v6.1.
- Performed unit testing to have defect free application code.
- Written JUnit Test cases while performing unit testing.
- Used MKS integrity as source control tool.
- Worked with QA team for testing and resolve defects.
- Used ANT automated build scripts to compile and package the application and implemented Log4j for the project.
ENVIRONMENT: Java 1.6, JSP, Servlets, XML, HTML5, CSS, Java Script, AJAX, JSON, jQuery, WebSphere Application Server v6.1, IBM RAD v8.0, ANT, JUnit, Oracle 10g, Sybase, MKS integrity
JAVA DEVELOPER
Confidential
RESPONSIBILITIES:
- Participated in all phases of full life-cycle development of the system, including requirements analysis, software design, software implementation, testing, deployment and documentation.
- Designed Use Case diagrams, Class diagrams and Sequence diagrams and Object Diagrams in the detailed design phase of the project using Rational Rose 4.0 .
- Developed presentation layer of the project using HTML, CSS, JSP 2.0, JSTL and JavaScrip t technologies.
- Developed code for modules of small to large scope.
- Developed complete Business tier using Stateless and Stateful Session beans with EJB 2.0 standards using WebSphere Studio Application Developer (WSAD 5.0).
- Written Form beans for the UI form validation and Action classes for the application flow.
- Involved in writing the code for server side validations using Struts validator Framework.
- Used various J2EE design patterns, like DTO, DAO , and Business Delegate, Service Locator, Session Facade, Singleton and Factory patterns.
- Involved in writing Hibernate mapping configuration files for POJO’s.
- Consumed Web Service for transferring data between different applications and involved in functionality enhancements of the Web Services.
- Used Object/Relational mapping Hibernate 3.0 framework as the persistence layer for interacting with Oracle 9i.
- Extensively used IBM WebSphere 5.0 while writing code and for creating Data Sources.
- Written complex SQL queries, stored procedures, functions and triggers in PL/SQL.
- Configured and used Log4J for logging all the debugging and error information.
- Developed Ant build scripts for compiling and building the project.
- Used IBM WebSphere 5.0 for deploying the application.
- Used CVS Repository for Version Control.
- Created test plans and JUnit test cases and test suite for testing the application.
- Provided production support and made several enhancements to application.
- Involved in unit integration, bug fixing, acceptance testing with test cases, code review.
ENVIRONMENT: Java 1.5, JSP 2.0, HTML, CSS, JavaScript, EJB 2.0, Struts 1.1, JDBC 2.0, IBM Web Sphere 5.0, XML, XSLT, XML Schema, JUnit 3.8.1, Rational Rose 4.0, Ant 1.5, UML, Hibernate 3, Oracle 9i and Windows.
JAVA DEVELOPER
Confidential
RESPONSIBILITIES:
- Involved in the design of MVC components.
- Involved in the design of SQL schemas.
- Developed view components using Struts Tlds and also created Actions classes and form beans.
- Involved in the development of JSPs those generate dynamic reports using Crystal Enterprise suit java API.
- Involved in developing parsers to parse the mails and store the information in database.
- Did setup of the debug environment against tomcat server, using remote debugging feature
- Developed the core module using s ervlets and Oracle 10g database.
- Implemented user interfaces using HTML and JSP .
- Installed and configured WebSphere 6.1 application servers
- Implemented the system using Eclipse, SQL Developer.
- Estimated the change requests and provided the implementation for the same.
ENVIRONMENT: Struts, JSP, Log4J, Tomcat, MySql and Eclipse, JBoss, HTML, Servlets, Web Sphere 6.1, Oracle 10g, Eclipse, Toad.