Hadoop Admin Resume
Addison, TX
SUMMARY
- Over 7+ years of IT professional experience with full project lifecycle development in J2EE technologies, Requirements analysis, Design, Development, Testing, Big Data, Deployment and production support of software applications.
- Experience in Design, Development, integration and implementation of web based applications, Client/server based applications, Hadoop, Mainframe and SOA based applications.
- Strong working experience in Insurance / Brokerage / Entertainment / Banking Financial Industries.
- Experience in Hadoop, Map Reduce techniques and HDFS.
- Hands on experience with Hadoop Eco system and programming with Java Map Reduce, Pig, Hive, HiveQL.
- Strong knowledge of Hbase, Sqoop, Flume, Oozie, Zoo Keeper.
- Strong Hands on experience in database development and with ETL tools.
- Extensive experience with both core and advanced Java technologies which includes Java, XML, JavaScript, JEE, JSP, EJB, JDBC, Struts, Servlets, JMS, JSF, JPA, Java API for XML Web Services, JCA, Spring and Hibernate.
- Experience in working with different operating systems Windows 98/NT/2000/XP/Vista/7/8, UNIX.
- Good experience in Shell Scripting, SQL Server, Unix and Linux, Open stock
- Extensively implemented IOC and Dependency Injection using Spring Frameworks.
- Extensive experience in using various design patterns such as MVC (Model - View-Controller), Singleton, Session Facade, DAO, and Business Delegate in the development of Multi-Tier distributed Enterprise Applications.
- Extensive knowledge in using Ajax Frameworks Prototype, EXTJ, jQuery, JSON, DOJO.
- Implemented SOA architecture with web services using SOAP, WSDL, UDDI and XML.
- Strong Hands on Experience with IBM MQ Series and JMS for communications between middleware components.
- Good experience with Spring, Hibernate 3.0, JDO, Agile Development and Performance Optimization.
- Hands on experience with various Application Servers and Web Servers including IBM Web Sphere, BEA Web logic, Jakarta Tomcat with wide variety of development tools like IBM WSAD/ RAD / RSA, My Eclipse, Dream Weaver.
- Having good RDBMS concepts knowledge and worked with MySQL, Oracle, Sybase, SQL Server, DB2.
- Proficient in SQL, PL/SQL developed Stored Procedures, Triggers, Functions and Packages besides writing DDL, DML and Transaction queries with development tools like TOAD and Rapid SQL.
- Experience with unit testing using JUnit and writing build scripts using ANT.
- Very good experience with SoapUI, Hudson, Jenkins, Mule server, Splunk logging and searching, TortoiseSVN, Subversion, git repositories, Clearcase.
- Worked on different OS UNIX (Sun Solaris, AIX), Linux, Windows.
- Good team player with ability to solve problems, organize and prioritize multiple tasks.
- Excellent communication and inter-personal skills.
TECHNICAL SKILLS
Operating Systems: DOS, Windows 98/XP/2000/7, Unix (Sun Solaris),Linux
Languages: Java, Java EE 1.5, Java Script, SQL, PL/SQL,C, C++
Web/Distributed Technologies: J2EE, Java EE 5, Servlets, JSP, Velocity, FreeMarker, Jprofiler,WSDL, UDDI, XML, JAX-RPC, JAX-WS, JSTL, EJB 1.1/2.0/3.0, JDBC 2.0/3.0, RMI, UML, MVC, JSF1.2/2.0, STRUTS, Spring, JAXP, JAXB, SAX, DOM, DOM4J, XML, XSL, XSLT, JMS, Hibernate, Pig, Hive, HiveQL, Sqoop, Flume, Oozie, Zoo Keeper, Java Map Reduce, MRUnit
Scripting: JSP 2.1, Java Script, HTML, DHTML, CSS, ANT, Unix Shell Scripting.
Open Frame works: Jakarta Struts, JSF, Hibernate 3.0, Spring, JSON, jQuery, Prototype, DOJO, Hadoop
RDBMS: ORACLE 8i/9i/10g, Sybase, MS-Access 2000, SQL-Server, Hbase
Web/Application Servers: WebSphere 5.1/6.0/7.0, Weblogic Application server, JBOSS, J2EE Server 1.4, Apache Tomcat 4.1/5.1/6.0/7.0, IBM HTTP Server, IIS
IDE / Tools: RAD 6.0/7.0/7.5, WSAD 5.1, Eclipse 3.1 with Eclipse 4.1.1, Rational Rose 98/2000
Methodologies: Unified Modeling Language (UML), Rational Unified Process (RUP), Agile
PROFESSIONAL EXPERIENCE
Confidential, Addison,TX
Hadoop Admin
Responsibilities:
- Installed Namenode, Secondary name node, Yarn (Resource Manager, Node managerApplication master), Data node.
- Installed and Configured HDP2.2
- Responsible for implementation and ongoing administration of Hadoop infrastructure.
- Monitored already configured cluster of 40 nodes.
- Installed and configured Hadoop components Hdfs, Hive, HBase.
- Communicating with the development teams and attending daily meetings.
- Addressing and Troubleshooting issues on a daily basis.
- Launched R - statistical tool for statistical computing and Graphics.
- Working with data delivery teams to setup new Hadoop users. This job includes setting up
- Linux users, setting up Kerberos principals and testing HDFS, Hive.
- Cluster maintenance as well as creation and removal of nodes.
- Monitor Hadoop cluster connectivity and security
- Manage and review Hadoop log files.
- File system management and monitoring.
- HDFS support and maintenance.
- Diligently teaming with the infrastructure, network, database, application and business intelligence teams to guarantee high data quality and availability.
Environment: Hadoop, MapReduce, HDFS, Pig, Hive, Kafka, Oozie, Cloudera CDH4, SQL, Hadapt, ZooKeeper, Linux, Java, J2EE, Web services, Oracle, DB2, LinuxShell Scripting, Red hat Linux, Bash Scripting.
Confidential, Somerset, NJ
Hadoop Developer
Responsibilities:
- Designing and creating stories for the development and testing of the application.
- Developing Java MapReduce programs for grouping the data and to calculate the algebraic calculations for the reducer.
- Configuring and performance tuning the sqoop jobs for importing the input (raw) data from the data warehouse.
- Developing hive and impala queries using partitioning, bucketing and windowing functions.
- Optimized hive joins for large tables and developed map reduce code for the full outer join of two large tables.
- Designed and developed entire pipeline from data ingestion to reporting tables.
- Creating the raw Avro data for an efficient feed to the map reduce processing.
- Design and Develop Pig Latin scripts and Pig command line transformations for data joins and custom processing of Map reduce outputs.
- Creating Hbase tables for random read/writes by the map reduce programs.
- Creating hive tables to the imported data for validation and debugging.
- Creating data model, schemas and stored procedures for reporting database.
- Creating Liquibase project for tracking and propagating reporting database changes.
- Designing and creating Oozie workflows to schedule and manage Hadoop, java, pig and sqoop jobs.
- Implemented custom workflow scheduler service to manage multiple independent workflows. Implemented a web application which uses Oozie Rest API and schedule jobs.
- Actively involved in developing front end spring web application for consumers to created custom profiles for data processing.
- Actively involved in deploying and testing the application in different environments.
- Configuring Hadoop Environment: Kerberos authentication, DataNodes, NameNodes, MapReduce, Hive, Pig, Sqoop, Oozie workflow engine
Environment: Hadoop, MapReduce, HDFS, Pig, Hive, Hbase, Oozie, Cloudera CDH4.5, Cloudera CDH5.1.3, Kerberos security, SQL, Linux, Java, J2EE, Web services, PostgreSQL, DB2, Netezza.
Confidential, MI
JAVA/J2EE Developer
Responsibilities:
- Designed and development applications using Rational Unified Process (RUP), UML and OOAD.
- Created Use cases, Class Diagrams, Sequence Diagrams using Rational Rose.
- Designed and developed the project using MVC design pattern.
- Developed user management screens using JSF framework, business components using Spring framework and DAO classes using Hibernate framework for persistence management and involved in integrating the frameworks for the project.
- Developed UI for Customer Service Modules and Reports using JSF, JSP’s, Apache MyFaces Components, Ajax4JSF framework (Ajax), JavaScript and Backing Beans.
- Implemented J2EE design patterns such as Session Facade, Factory, DAO, DTO, MVC.
- Designed & Developed persistence service using Hibernate framework.
- Hibernate used as the ORM mapping tool and worked with Hibernate Query Language (HQL).
- Wrote Hibernate configuration file, Hibernate mapping files and define persistence classes to persist the data into Oracle Database.
- Worked on advanced Hibernate associations with multiple levels of Caching, lazy loading.
- Involved in writing JSF Backing Beans for controlling the UI and writing custom components and validators.
- Implemented Ajax using Ajax4JSF Framework and nested taglib API along with other standard taglibs in the application, as a better approach for screen development and reusability of visual components in the JSP pages.
- Implemented business logic in Middle Tier using pluggable components, which were specified in the customized xml documents using Spring framework.
- Implementing Session facade and data communication from Web Services channel.
- Developed Data Access Objects to access middleware Web Services.
- Configured and Integrated JSF, Spring and Hibernate frameworks.
- Responsible for writing Java code to convert HTML files to PDF file using Apache FOP.
- Developed Message Driven Beans (MDBs) for receiving and processing data from IBM MQ Series using JMS.
- Rational ClearQuest used for defects, change tracking across the application.
- Designed and developed direct live interfaces to handle multiple systems within the bank to subscribe the data in real-time.
- Involved in the performance tuning of PL/SQL statements.
- Developed database triggers and procedures to update the real-time cash balances.
- Worked closely with the testing team in creating new test cases and also created the use cases for the module before the testing phase.
- Wrote ANT build scripts to compile Java classes and create jar, performed unit testing and package them into ear files.
- Coordinated work with DB team, QA team, Business Analysts and Client Reps to complete the client requirements efficiently.
Environment: Java/J2EE, JMS, JNDI, JSP, JSF, MyFaces, Spring, Tiles, Hibernate, HTML, DHTML, IBM MQ Series, CSS, WebSphere 5.1.2, Ant, ClearQuest, Oracle 9i, AJAX, JSTL, Eclipse, JUnit, JavaScript, VSS.
Confidential, Webster, NY
JAVA/J2EE consultant
Responsibilities:
- Designed and developed the application using Agile methodology and followed TDD, Scrum, pair programming.
- Wrote Technical design document with the class, sequence, activity diagrams in each use case.
- Involved in design and implementation of MVC architecture in different modules.
- Integrated business layer using Hibernate with Spring DAO.
- Implemented Spring Bean Factory to create proxied object using AOP framework
- Designed and Developed the UIFramework built using JSF Apache My Faces, Ajax4JSF.
- Involved in writing JSF Backing Beans for controlling the UI and writing custom components and validators.
- Implemented JSF Converters to handle formatting and Localization and configured the faces-config.xml to create a web application navigation rule using Spring Webflow.
- Implemented J2EE design patterns such as Session Facade, Factory, DAO, DTO and Proxy.
- Used Asynchronous JavaScript and XML (AJAX) for better and faster interactive Front-End.
- Developed various reusable Helper and Utility classes which are used across all the modules of the application.
- Designed and developed Business components and Integrated with Spring framework.
- Wrote data Components for Plan Setup, wrote and called StoredProcedures using Java, HQL, Hibernate.
- Involved in configuring and integrating JSF, Spring and Hibernate frameworks.
- Implemented SOA architecture with Web Services using SOAP, WSDL, UDDI and XML.
- Designed & developed LDAP Authentication & Authorization Service using Java, LDAP, JNDI.
- Used Jprobe to identify performance bottlenecks and fix the issues.
- Involved in transforming XML data in to Java Objects using a JAXB binding tool.
- Responsible for configuring and deploying the builds on WebSphere App Server
- Implemented asynchronous messaging service using JMS and Message Driven Bean’s.
- Implemented ILog JRules a Business Management Rules System that enables business and IT users to write and maintain the logic applied by applications that automatically implement decisions.
- Wrote automation scripts for invoking, monitoring the batch jobs using UNIX Shell scripts.
- Wrote notification service that executes daily, sends emails and escalations to participants.
- Involved in Unit & Integration Testing for different Modules in the application.
- Wrote build & deployment scripts using ANT, Maven, shell scripts and JACL on unix env.
- Did the application support on different environments from QA to Production. Was responsible for analyzing the issues, providing technical & functional support to the users and fix them.
Environment: Java 5.0, JSP, Ajax, Ajax4JSF, JSF, Apache MyFaces, RichFaces, Spring Frame Work, Hibernate, JMS, Apache Axis 2.0, LDAP, JNDI, UDDI, WSDL, SOA, SOAP, XML, HTML, Log 4j, WebSphere 6, RAD 7, Oracle, SunOS (UNIX), Shell Script, Sub Version, Ant and WindowsXP.
Confidential
JAVA/J2EE Consultant
Responsibilities:
- Used OOAD for building applications.
- Involved in gathering business requirements, analyzing the project and creating use Cases and Class Diagrams.
- Interacted coordinated with the Design team, Business analyst and end users of the system.
- Created sequence diagrams, collaboration diagrams, class diagrams, usecases and activity diagrams using Rational Rose for the Configuration, Cache & logging Services.
- Designed and developed the project using MVC design pattern.
- Developed front end using Struts framework, business components using EJB’s.
- Implementing Struts/ Tiles based framework to present the data to the user.
- Created the web UI using Struts, JSP, Servlets and Custom tags.
- Deployment of Application in the JBoss Server environment using ANT tool using Struts architecture with the JSP Client.
- Configured Struts Dyna ActionForms, MessageResources, ActionMessages, ActionErrors, Validation.xml, and Validator-rules.xml.
- Designed and developed Caching and Logging service using Singleton pattern, Log4j.
- Coded different action classes in struts responsible for maintaining deployment descriptors like struts-config, ejb-jar and web.xml using XML.
- Designed and Developed front-end using struts framework. Used JSP, JavaScript, JSTL, EL, Custom Tag libraries, Tiles and Validations provided by struts framework.
- Wrote authentication and authorization classes and manage it in the frontcontroller for all the users according to their entitlments.
- Developed and deployed Session Beans and Entity Beans for database updates.
- Implemented caching techniques, wrote POJO classes for storing data and DAO’s to retrieve the data and did other database configurations using EJB 3.0.
- Developed stored procedures and complex packages extensively using PL/SQL and shell programs.
- Used Struts-Validator frame-work for all front-end Validations for all the form entries.
- Developed SOAP based Web Services for Integrating with the Enterprise Information System Tier.
- Design and development of JAXB components for transfer objects.
- Prepared EJB deployment descriptors using XML.
- Involved in Configuration and Usage of Apache Log4J for logging and debugging purposes.
- Wrote Action Classes to service the requests from the UI, populate business objects & invoke EJBs.
- Used JAXP (DOM, XSLT), XSD for XML data generation and presentation
- Wrote Junit test classes for the services and prepared documentation.
Environment: Java, JSP, Servlets, Struts, JAXB, XSLT, DOM, XML, EJB 3.0, WebServices, JNDI, J2EE, Eclipse, JBoss, Ratonal Rose, CVS, Oracle 9i, Junit, Rational Rose, JavaScript, DHTML.
Confidential
JAVA/ J2EE Consultant
Responsibilities:
- Designed and developed applications using RUP Methodology, UML and OOAD.
- Created Use cases, Class Diagrams, Sequence Diagrams using Rational Rose.
- Wrote Action classes, Action Forms Beans, Validators for Policy Change Details Components.
- Developed JSP’s and HTML for front end User Interface.
- Implemented J2EE Design Patterns like MVC, Service Locator and Session Façade.
- Designed, developed and implemented the business logic required for Security presentation controller.
- Configured JMS in the WebSphere Application server to send messages to all the donors.
- Wrote Message Handler Adapter for enterprise calls (Asynchronous Messages) using Message Driven Beans, JMS, and XML.
- Wrote SQL queries, used JDBC to connect to Oracle database for larger data retrievals.
- Involved in preparation of docs like Functional Specification document and Deployment Instruction documents.
- Wrote Ant Scripts for building the applications and deployments.
- Involved in development of Business components like Session Beans and Entity Beans for Transactions and business logic purpose.
- Used LDAP for user security credentials authentication and for validating the user roles.
- Developed MessageHandler Adapter, which converts the data objects into XML message and invoke an enterprise service and vice-versa using JAVA, JMS, MQ Series.
- Wrote Junit test cases for testing the enhancement modules.
Environment: RUP, UML, EJB 2.x, JNDI, Servlets, JSP, JMS, MQSeries, XML, J2EE, JDBC, WSAD, WebSphere, LDAP, ANT, Rational Rose, Dream Viewer, CVS, TestDirector, Oracle, Junit, JavaScript, DHTML.
