- 8+ years of overall experience with strong emphasis on Design, Development, Implementation, Testing and Deployment of Software Applications inHadoop,
- Hadoop Developer with 3+ years of working experience on designing and implementing complete end - to-end HadoopInfrastructure using MapReduce, PIG, HIVE.
- Java Programmer with 5+ years of Extensive programming experience in developing web based applications and Client-Server technologies using Java, J2EE.
- Extensive experience in HDFS, MapReduce,Pig, Hive, Sqoop, Yarn, ETL and RDBMS, extensive development experience using Java, J2EE, JSP, Servlets, Spring and Hibernate.
- Experienced with Agile SCRUM methodology, involved in design discussions and work estimations, takes initiatives, very proactive in solving problems and providing solutions.
- Experience in different layers of HadoopFramework - Storage (HDFS), Analysis (Pig and Hive), Engineering (Jobs and Workflows).
- Good Experience in understanding the client's Big Data business requirements and transform it into Hadoop centric technologies.
- Experience in using Cloudera Manager for installation and management of single-node and multi-node Hadoopcluster (CDH3, CDH4 & CDH5).
- Experience in installing Hadoopcluster using different distributions of Apache Hadoop, Cloudera and Hortonworks.
- Worked with Sqoop in Importing and exporting data from different databases like MySql, Oracle into HDFS.
- Responsible for writing MapReduce programs.
- Hands on NoSQL database experience with HBase.
- Experience in developing Pig scripts and Hive Queries.
- Written Hive queries for data analysis and to process the data for visualization.
- Experience in managing and reviewing HadoopLog files.
- Experience in deploying and managing the multi-node development, testing and production.
- Experience in designing and coding web applications using Core Java and J2EE Technologies- JSP, Servlets and JDBC.
- Excellent knowledge in Java and SQL in application development and deployment.
- Hands on experience in creating various database objects like tables, stored procedures, functions, and triggers using SQL, PL/SQL,DB2.
- Expert knowledge over J2EE Design Patterns like MVC Architecture, Front Controller, Session Facade, Business Delegate and Data Access Object for building J2EE Applications.
- Hands-on experience in using the Integrated Development Environments such as Eclipse, Edit Plus, Net Beans, Rational Application Developer (RAD), IntelliJ IDEA.
- Experience working with web services using SOAP, UDDI, WSDL, REST and implementation using APACHE AXIS and APACHE CXF.
- Experienced in using CVS, SVN and Sharepoint as version manager.
- Proficient in unit testing the application using Junit, MRUnit and logging the application using Log4J.
- Received appreciations from the clients for the efforts and intuitiveness.
- Excellent communication, interpersonal and analytical skills and a highly motivated team player withthe ability to work independently.
- Ability to learn and adapt quickly to the emerging new technologies and paradigms
Hadoop Ecosystem: Hadoop 2.2, HDFS, MapReduce, Pig, Hive, Sqoop, Zookeeper, Yarn, Spark, Impala, Kafka, Tez, Hbase, Cassandra
Hadoop Management & Security: Hortonworks, Ambari, Cloudera Manager, Apache Knox, XA Secure
Server Side Scripting: UNIX Shell Scripting
Database: Oracle 10g, Microsoft SQL Server, MySQL, DB2, SQL, Sybase, RDBMS.
Programming Languages: Java, J2EE, JSTL, JDBC 3.0/2.1, JSP 1.2/1.1, Java Servlets, JMS, JUNIT, Python, Perl
Web Servers: Apache Tomcat 5.x, BEA WebLogic 8.x, IBM WebSphere 6.0/5.1.1
NO SQL Databases: HBase, Cassandra, Mongo DB
OS/Platforms: Mac OS X 10.9.5, Windows2008/Vista/2003/XP/2000/NT, Linux, Unix
SDLC Methodology: Agile (SCRUM), Waterfall.
Design: UML, Rational Rose, E-R Modeling.
Build Tools: ANT and Maven.
Version Control System: CVS, SVN, GITHUB.
Confidential, Bethesda, MD
Sr. Hadoop Developer
- Gathered the business requirements from the Business Partners and Subject Matter Experts.
- Installed Hadoop Ecosystem components. Involved in monitoring and managing Hadoop cluster using Cloudera manager. Supported Map Reduce Programs those are running on the cluster.
- Installed and configured Pig and also developed Pig Latin scripts.
- Managed and reviewed the Hadoop log files (both server logs and agent logs).
- Responsible to manage and load data into Hadoop cluster coming from different sources.
- Created Hive tables and worked on them using Hive QL.
- Used PDI (Pentaho Data Integration) to extract, transform and load (ETL) using metadata driven approach. Configured and managed Pentaho business analytics and PDI tools.
- Used H-Base for random access of data from H-Base database (column oriented database).
- Imported data frequently from MySQL to HDFS using SQOOP.
- Involved in HDFS maintenance and loading of structured and unstructured data.
- Worked on Puppet configuration management and automation.
- Monitored and imported preexisting OOZIE work flow for pig and Hadoop jobs.
- Developed Scripts and Batch Job to schedule various Hadoop Program.
- Created monthly and quarterly audit reports for tickets raised to tickets closed for administration team.
- Communicated with developers and other teams when Root cause analysis need to be performed.
- Ensure test data and environment is ready for testing.
- Created Quality Metrics reports for testing team. Created defect close rate and test status reports and facilitated test status meetings.
Environment: Hadoop, HDFS, Map Reduce, Pig, Hive, H-base, SQOOP, OOZIE, Cloudera Manager, Flume, Zookeeper and Linux Ubuntu operating system, HPQC, Spark, Storm, Kafka, Java, UNIX, MySQL, Maven.
Confidential, Baltimore MD
Sr. Hadoop Developer
- Implemented data ingestion pipeline into Hadoop.
- Implemented Flume, Hive, Pig, Sqoop and Oozie for data ingestion.
- Worked on developing Map reduce programs for data processing.
- Experienced in working with Sqoop to import and export data from HDFS.
- Implemented Hive UDFs to solve and improve performance.
- Worked on Parquet files in Impala, Implemented multiple batch query in Impala.
- Automated daily, weekly, monthly jobs using Oozie.
- Experienced shell scripting and Python.
- Experienced in rewriting existing python modules to deliver certain format of data.
- Involved in designing data visualization to present current impact and growth.
- Worked on running Hadoop streaming jobs in order to process terabytes of xml format data.
- Created Hive tables, worked on loading data into hive tabled and Experienced in loading and transforming heavy sets of structured, semi structured and unstructured data.
- Involved in defining job flows.
- Extracted files from multiple databases using Sqoop and stored and created hive tables.
- Used Kafka for streaming data along with Spark.
- Responsible for managing and reviewing log files.
- Proficient work experience with NOSQL, HBase.
- Experienced in loading data from UNIX file system to HDFS.
- Responsible to manage data coming from different sources.
Environment: Hadoop, HDFS, Pig, Hive, Sqoop, Flume, Map Reduce, Oozie, Java 6/7, Oracle 10g, YARN, UNIX Shell Scripting, SOAP, REST services, Oracle 10g, Agile Methodology, JIRA, Auto Sys.
Senior Java J2EE Developer
- Designed and development applications using Rational Unified Process (RUP), UML and OOAD.
- Created Use cases, Class Diagrams, Sequence Diagrams using Rational Rose.
- Designed and developed the project using MVC design pattern and integrated Spring, JSF and Hibernate.
- Developed user management screens using JSF framework, business components using Spring framework and DAO classes using Hibernate framework for persistence management and involved in integrating the frameworks for the project.
- Designed & Developed ORM persistence layer using Hibernate framework.
- Created Hibernate configuration file,Hibernate mapping files and defined persistence classes to persist the data into MySQL Database.
- Worked on advanced Hibernate associations with multiple levels of Caching, lazy loading.
- Involved in writing JSF Backing Beans for controlling the UI and writing custom components and validations.
- Experience in developing web application by using AJAX.
- Implemented Ajax using Ajax4JSF Framework and nested taglibs API along with other standard taglibs in the application, as a better approach for screen development and reusability of visual components in the JSP pages.
- Implemented Relational Database Management Systems RDBMS.
- Implemented business logic in Middle Tier using pluggable components, which were specified in the customized xml documents using Spring framework.
- Responsible for writing Java code to convert HTML files to PDF file using Apache FOP.
- Developed Message Driven Beans (MDBs) for receiving and processing data from IBM MQ Series using JMS.
- Rational Clear Quest used for defects, change tracking across the application.
- Experience in transmitting the data using JSON and Web services
- Designed and developed direct live interfaces to handle multiple systems within the bank to subscribe the data in real-time.
- Wrote build & deployment scripts using ANT on Linux environment and deployed the application on web logic server.
- Developed the Windows andweb based Customized GIS software for utility applications of Land Information Systems
- Worked closely with the testing team in creating new test cases and also created the use cases for the module before the testing phase.
- Wrote ANT build scripts to compile Java classes and create jar, performed unit testing and package them into ear files.
- Coordinated work with DB team, QA team, Business Analysts and Client Reps to complete the client requirements efficiently.
Confidential, Hartford, CT
- Involved in the analysis, design, and development and testing phases of Software Development Lifecycle (SDLC).
- Used Spring 2.5 Framework for DI/IOC and ORM components to support the Ibatis tool.
- Used Core Java classes to implement business logic.
- Worked on XML parsing by using JAXB for XML Schema Validation.
- Used XSLT to transform my XML data structure into HTML pages.
- Consumed SOAP based web services to transfer information to the Sears server from List server.
- Used Hibernate and JPA ORM in the back end to fetch data from Oracle databases
- Used Spring JDBC and Hibernate Session Factory for DB Transaction Management.
- Developed JUnit test cases for unit test of all developed modules,,
- Maven was used to manage dependencies in the application and deploy it on Websphere Application server. PVCS was used for the version control.
- Deployed and tested Applications on WAS.
- Maven was used as Project Management tool.
- Involved in Analysis, Design, Coding and Development of custom Interfaces.
- Involved in both maintenance and new enhancements of the application
- Developed Servlets and JDBC were used in retrieving data.
- Deployed EJB Components on WebLogic.
- Tested the modules and fixed the bugs.
- XML was used to transfer the data between different layers
- Developed presentation layer using JSP, HTML and CSS
- Dealt with java Beans helper classes and Servlets for interacting with the user
- Worked on database interaction layer for insertions, updating and retrieval operations on data.