- Over 7+ years of professional IT experience which includes experience in Bigdata, Hadoop ecosystem related technologies in Banking, Insurance and Communication sectors.
- Well versed in installation, configuration, supporting and managing of Big Data and underlying infrastructure of Hadoop Cluster.
- Hands on experience on major components in Hadoop Ecosystem like Hadoop Map Reduce, HDFS, HIVE, PIG, Hbase, Zookeeper, Sqoop, Oozie, Flume and Avro.
- Excellent understanding and knowledge of NOSQL databases like MongoDB, HBase, Cassandra, and SOLR/Lucene.
- Responsible for setting up processes for Hadoop based application design and implementation.
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice - versa.
- Good understanding of Data Structures and Algorithms.
- Experience in managing and reviewing Hadoop log files.
- Very good experience in complete project life cycle (design, development, testing and implementation) of Client Server and Web applications.
- Experience in Object Oriented Analysis, Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns and Core Java design patterns.
- Experience in managing Hadoop clusters using Cloudera Manager tool.
- Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
- Extensive expertise on development using SQL and PL/SQL
- Extensive expertise working in Oracle, DB2, SQL Server and My SQL database.
- Experience in Java, JSP, Servlets, EJB, WebLogic, WebSphere, Hibernate, Spring, JBoss, JDBC, RMI, Java Script, Ajax, Jquery, XML, and HTML
- Knowledge on Unix and Database administration.
- Determined, committed, hardworking with strong communication, interpersonal and organizational skills.
- Ability to work in a team and coordinate/resolve issues with team of developers and other stakeholders.
Big Data/Hadoop: : HDFS, Hadoop MapReduce, Zookeeper, Hive, Pig, Sqoop, Flume, Oozie.
Methodologies: : Agile, V-model, Waterfall model
Databases: : HBase, MongoDB, Cassandra, Oracle 10g,11g, MySQL, Couch, MS SQL server
Web Tools/Frameworks: : HTML, Java Script, XML, ODBC, JDBC, Java Beans, EJB, MVC, Ajax, JSP, Servlets, Struts, Junit, REST API, Spring, Hibernate
Web/Application Server: : Apache, Tomcat, OC4J, Web Logic and Sun one Webserver.
Confidential, Chicago, IL
- Worked on analysing Hadoop cluster and different big data analytic tools including Pig, Hive Hbase database and Sqoop
- Responsible for building scalable distributed data solutions using Hadoop
- Installed and configured Flume, Hive, Pig, Sqoop, HBase on the Hadoop cluster.
- Managing and scheduling Jobs on a Hadoop cluster using Oozie.
- Implemented nine nodes CDH4 Hadoop cluster on Ubuntu LINUX.
- Worked on installing cluster, commissioning & decommissioning of datanode, namenode recovery, capacity planning, and slots configuration.
- Setup Hadoop cluster on Amazon EC2 using whirr for POC.
- Resource management of HADOOP Cluster including adding/removing cluster nodes for maintenance and capacity needs
- Involved in loading data from UNIX file system to HDFS.
- Created HBase tables to store variable data formats coming from different portfolios.
- Implemented various requirements using Pig scripts.
- Implemented test scripts to support test driven development and continuous integration.
- Responsible to manage data coming from different sources.
- Installed and configured Hive and also implemented various business requirements by writing Hive UDFs.
- Experienced on loading and transforming of large sets of structured, semi structured and unstructured data.
- Cluster coordination services through Zookeeper.
- Experience in managing and reviewing Hadoop log files.
- Exported the analysed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
- Analysed large amounts of data sets to determine optimal way to aggregate and report on it.
- Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.
Environment: Hadoop, HDFS, Hive, Flume, HBase, Sqoop, PIG, Java (JDK 1.6), Eclipse, MySQL and Ubuntu, Zookeeper.
- Involved in review of functional and non-functional requirements.
- Installed and configured Hadoop MapReduce, HDFS, Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
- Installed and configured Pig and also written PigLatin scripts.
- Wrote MapReduce job using Pig Latin.
- Has solid understanding of REST architecture style and its application to well performing web sites for global usage.
- Involved in ETL, Data Integration and Migration
- Imported data using Sqoop to load data from Oracle to HDFS on regular basis.
- Developing Scripts and Batch Job to schedule various Hadoop Program.
- Written Hive queries for data analysis to meet the business requirements.
- Creating Hive tables and working on them using Hive QL.
- Importing and exporting data into HDFS from Oracle Database and vice versa using Sqoop.
- Experienced in defining job flows.
- Got good experience with NOSQL database HBase, MongoDB & Cassandra.
- Hybrid implementation using Oracle and MongoDB.
- Involved in creating Hive tables, loading the data and writing hive queries that will run internally in a map reduce way.
- Developed a custom FileSystem plugin for Hadoop so it can access files on Data Platform.
- The custom FileSystem plugin allows Hadoop MapReduce programs, HBase, Pig and Hive to work unmodified and access files directly.
- Designed and implemented Mapreduce-based large-scale parallel relation-learning system
- Extracted feeds form social media sites such as Facebook, Twitter using Python scripts.
- Setup and benchmarked Hadoop/HBase clusters for internal use
Environment: Hadoop, MapReduce, HDFS, Hive, Java, Hadoop distribution of Cloudera, Pig, HBase, Linux, XML, Java 6, Eclipse, Oracle 10g,11, PL/SQL, MongoDB, Toad.
Confidential, Madison, WI
- Processed data into HDFS by developing solutions, analyzed the data using MapReduce, Pig, Hive and produce summary results from Hadoop to downstream systems
- Used Sqoop widely in order to import data from various systems/sources (like MySQL) into HDFS
- Applied Hive quires to perform data analysis on HBase using Storage Handler in order to meet the business requirements
- Created components like Hive UDFs for missing functionality in HIVE for analytics.
- Hands on experience with NoSQL databases like HBase, Cassandra for POC (proof of concept) in storing URL’s and images.
- Developing Scripts and Batch Job to schedule a bundle (group of coordinators) which consists of various Hadoop Programs using Oozie
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
- Worked with cloud services like Amazon web services (AWS)
- Involved in ETL, Data Integration and Migration
- Used different file formats like Text files, Sequence Files, Avro
- Cluster co-ordination services through Zookeeper
- Assisted in creating and maintaining Technical documentation to launching HADOOP Clusters and even for executing Hive queries and Pig Scripts
- Assisted in Cluster maintenance, cluster monitoring, adding and removing cluster nodes and Troubleshooting
- Installed and configured Hadoop, Mapreduce, HDFS, Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
Environment: MapReduce, HDFS Sqoop, Flume, LINUX, Oozie, Hadoop, Pig, Hive, Hbase, Cassandra, Hadoop Cluster, Amazon Web Services
Senior Java/J2EE Developer
- Involved in requirements gathering, analysis, and design and in development.
- Involved in developing Webservices using SOAP, WSDL and UDDI Components.
- Involved in communicating with end client while requirements gathering and demo of project.
- Involved in Use Case Realization, Use Case Diagrams, Sequence Diagrams and Class Diagram for various modules.
- Worked with BA in requirement Analysis and prepared detailed software requirement document.
- Involved in Coding, Debugging & Code review.
- Developed Web Service Client programs using JAX-RPC.
- Used Spring Inversion of Control (IOC) to wire DAO and delegate object to the registry.
- Design and implementation of domain model layer (used by application for DB interaction) using Spring and Hibernate.
- Used struts framework for web tier and spring MVC on back end code.
- Implemented Hibernate for persisting Java objects, JUNIT or Spring AOP to test performance.
- Involved in writing ANT Scripts for building the web application. Used SVN for version control of the code and configuration files. Log4j was used to log the events in the application.
- Involved in customizing Ajax calls, types and strategies using Dojo tool kit.
- Implemented DB Connector using Singleton Pattern.
- Followed RUP Process and used VSS for version control and used JUNIT for unit testing.
- Involved In writing JUnit test cases using JUnit framework
- Involved in Units integration, bug fixing, and User acceptance testing with test cases.
- Involved in different phases to gather requirements, document the functional specifications, design, data modeling and development of the applications.
- J2EE Front-End and Back-End supporting business logic, integration, and persistence.
- Used JSP with Spring Framework for developing User Interfaces.
- Integrated Security Web Services for authentication of users.
- Used Hibernate Object/Relational mapping and persistence framework as well as a Data Access abstraction Layer.
- Data Access Objects (DAO) framework is bundled as part of the Hibernate Database Layer.
- Designed Data Mapping XML documents that are utilized by Hibernate, to call stored procedures.
- Responsible for Testing and moving the application into Staging and Production environments.
- Responsible for Project Documentation, Status Reporting and Presentation.
- Used CVS version control to maintain the Source Code.
- Involved in evaluating the existing system
- Involved in designing and development of the SSO, stabilizing the system (meeting Performance requirements, functionality implementation), providing support and enhancing the application according to business needs..
- Fixed the bugs and solved the issues raised.
- Made minor & major enhancements as & when required.
- Prepared the Gap Analysis Docs
- Involved in the fixing of existing bugs.
- Helped the team in understanding the requirements.
- Involving in Unit, Integration and Pre-Production testing of the application
- Prepared the GAP Analysis docs.
Environment:: Java, JSP, Servlets, JDBC, DOJO, HTML, CSS.