SUMMARY

• Having 8+ years of experience on Java, J2EE, Big Data Technologies, working knowledge in Hadoop and its stack including big data analytics and expertise in application Design and Development in various domains with an emphasis on Data warehousing tools using industry accepted methodologies and procedures.
• 3+ years of experience in Hadoop Developer/Administration and four years of java application development.
• Experienced Hadoop Developer, have a strong background with file distribution systems in a big - data arena.
• Understands the complex processing needs of big data and have experience developing codes and modules to address those needs.
• Experience in using Pig, Hive, Scoop, Oozie, Ambari and Cloudera Manager.
• Experience working with Hive data warehouse system, developing data pipelines, implementing complex business logic and optimizing hive queries.
• Design and implementation of business logic and data processing routes using Apache Camel.
• Good knowledge of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, Resource Manager, Node Manager, Application Master and Map Reduce concepts.
• Experience with complex data processing pipelines, including ETL and data ingestion dealing with unstructured and semi-structured data
• Good knowledge on designing and implementing ETL process to load data from various data sources to HDFS using Flume and Sqoop, performing transformation logic using Hive, Pig and integration with BI tools for visualization/reporting
• Extensively used ETL methodology for supporting of Extract, Transform, and Load environment.
• Well versed in installation, configuration, supporting and managing of Big Data and underlying infrastructure of Hadoop Cluster along with CDH3&4 clusters.
• Experience in installing, configuring, and administrating Hadoop cluster for various distributions like Apache, Cloudera, Map R and Horton works 2.2.
• Very good understanding on NOSQL databases like mongo DB, HBase and Cassandra.
• Knowledgeable of Spark and Scala mainly in framework exploration for transition from Hadoop/Map Reduce to Spark.
• Experience in using Apache Flume for collecting, aggregating and moving large amounts of data from application servers.
• Hands-on experience with message brokers such as Apache Kafka, IBM WebSphere, and RabbitMQ.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
• Generated Java APIs for retrieval and analysis on No-SQL database such as HBase and Cassandra.
• Analyzed data on Cassandra cluster by running queries for searching, sorting and grouping.
• Knowledge on coding skills Kafka, Scalaand Spark.
• Knowledge of Data Lakes,Processing with Graph DBs and Scala.
• Hands on experience in RDBMS and Linux shell scripting.
• Extending Hive and Pig core functionality by writing custom UDFs.
• Experience in analyzing data using HiveQL, Pig Latin and Map Reduce.
• Developed Map Reduce jobs to automate transfer of data from HBase.
• Knowledge in job work-flow scheduling and monitoring tools like oozie and Zookeeper.
• Expertise in RDBMS like MS SQL Server, MySQL, Greenplum,DB2 and NoSql like Cassandra
• Knowledge of job workflow scheduling, monitoring and of administrative tasks such as installing Hadoop, Commissioning and decommissioning, and its ecosystem components such as Flume, Oozie, Hive and Pig.
• Excellent Java development skills using J2EE, Spring, J2SE, Servlets, JUnit, MRUnit, JSP, JDBC.
• Knowledge in Software Development Life Cycle (Requirements Analysis, Design, Development, Testing, Deployment and Support).
• Experience inwriting Complex SQL Queries involving multiple tablesinner and outer joins.
• Experienceinoptimizing the queries by creating variousclustered, non-clustered indexesand indexed views using and data modelingconcepts.
• Experience with Oracle 9i -PL/SQL programming and SQL * Plus.
• Techno-functional responsibilities include interfacing with users, identifying functional and technical gaps, estimates, designing custom solutions, development, leading developers, producing documentation, and production support.
• Excellent interpersonal and communication skills, creative, research-minded, technically competent and result-oriented with problem solving and leadership skills.

TECHNICAL SKILLS

Programming Languages: C,C++,Java, Shell Scripting,PL/SQL

J2SE/J2EE Technologies: Java, J2EE, Servlets, JSP, Java Beans, JDBC, Struts 2.0, EJB, Springs, Hibernate, JTA, JMS, Web Services.

IDE’s: RAD 6.0, Eclipse 3.1 with My Eclipse 4.1.1, Rational Rose 98/2000,Net beans

Big Data Ecosystem: HDFS, Map Reduce, Yarn, Hive, Pig,Sqoop, Impala, Kafka, Oozie, Spark, Zookeeper, Flume, Storm AWS, EC2, EMR

Web Technologies: HTML, DHTML, XHTML, CSS, Java Script, My Faces, Rich Faces, JSF, PHP and AJAX.

Monitoring Tools: Ganglia, Nagios, Cloud era Manager, Ambari

Security: Kerberos, Sentry

Xml Technologies: XML, XSL, XQuery, XSD and XSLT.

Operating Systems: HP-UX, LINUX, Windows 9X/2000/XP

Databases: Oracle 10g/9i, MySQL, DB2, MS-SQL Server

Web Servers: Web Sphere 5.1/6.0, Web logic Application server, JBOSS, J2EE Server 1.4, Apache Tomcat 4.1/5.1, IBM HTTP Server.

Methodologies: Unified Modeling Language (UML), Rational Unified Process (RUP), Agile.

PROFESSIONAL EXPERIENCE

Confidential - O’Fallon, MO

Hadoop Developer/Admin

Responsibilities:

• Creating the automated build and deployment process for application, re-engineering setup for better user experience, and leading up to building a continuous integration system.
• Predicted consumer behaviour, such as what products a particular user's has bought and made predictions/recommendations based on recognizing patterns by using Hadoop, HiveandPig queries.
• Installed and configured Hadoop, MapReduce, and HDFS.
• Developed multiple MapReduce jobs using JavaAPI for data cleaning and pre-processing.
• Importing and exporting data into HDFS and HIVE from an Oracle 11g database using Sqoop
• Responsible to manage data coming from different sources
• Monitoring the running MapReduce programs on the cluster.
• Responsible for loading data from UNIX file systems into HDFS.
• Installed and configured Hive.
• Worked with application teams to install Hadoop updates, patches, version upgrades as required.
• Installed and configured Hive, Pig,Sqoop and Oozie on the HDP 2.0 cluster.
• Involved in implementing High Availability and automatic failover infrastructure to overcome single point of failure for Namenode utilizing zookeeper services.
• Implemented HDFS snapshot feature.
• Installed Apache Tez, a programing framework which is built on YARN in increase performance.
• Experience on deployment of Apache Tez on top of YARN.
• Experience in Migrating Business reports to Spark, Hive, Pig and Map Reduce.
• Performed a Major upgrade in production environment from HDP 1.3 to HDP 2.0.
• Worked with big data developers, designers and scientists in troubleshooting map reduce job failures and issues with Hive, Pig and Flume.
• Involved in Installation and configurations of patches and version upgrades.
• Involved in HadoopCluster environment administration that includes adding and removing cluster nodes, cluster capacity planning, performance tuning, clusterMonitoring.
• Troubleshooting.
• Supported MapReduce Programs those are running on the cluster.
• Involved in HDFS maintenance and administering it through Hadoop-JavaAPI
• Involved in creating Hive Tables, loading with data and writing Hive queries which will invoke and run MapReduce jobs in the backend.
• Installed and configured Pig.
• Experienced in BigData technologies such as Hadoop, Cassandra, Presto, Spark, Flume, Storm, AWS, SQL, etc
• Wrote Pigscripts to process unstructured data and create structure data for use with Hive.
• Developed the Sqoopscripts to order and make the interaction between Pig and MySQL Database.
• Developed scripts and automated data management from end to end and sync up b/w all the clusters.

Environment: Apache Hadoop 2.0.0, Pig 0.11, Hive 0.10, Sqoop 1.4.3, Flume, MapReduce, HDFS, LINUX, Oozie, Cassandra, Tez, Hue, HCatalog, Java, Eclipse, VSS, Red Hat Linux.

Confidential - CedarRapids, Iowa

Big Data/Hadoop Developer

Responsibilities:

• Responsible for building scalable distributed data solutions usingHadoop 2.6.3
• Imported data using Sqoop1.99.6-rc2 to load data from MySQL to HDFS on regular basis from various sources.
• Written multiple MapReduce 2.0programs to power data for extraction, transformation and aggregation from multiple file formats including XML, JSON 1.8.3, CSV& other compressed file formats.
• Reviewed the HDFS usage and system design for future scalability and fault-tolerance.
• Worked on python 3.5.1 scripts to analyze the data of the customer.
• Loaded and transformed large sets of structured, semi structured and unstructured data in various formats like text, zip, XML and JSON.
• Defined job flows and developed simple to complex MapReduce2.0 jobs as per the requirement.
• Developed MapReduce jobs in Python for data cleaning and data processing.
• Developed PIGUDFs for manipulating the data according to Business Requirements and also worked on developing custom PIG0.15.0 Loaders.
• Migration ofInformaticsjobs toHadoopSqoop jobs and load into oracle database.
• Used Spark 0.15.0 API overHadoopYARN to perform analytics on data in Hive 1.1.1.
• Efficiently put and fetched data to/from HBase 1.1.2by writing MapReduce job.
• Responsible for creating Hive1.1.1 tables based on business requirements.
• Involved in design Cassandra3.0 data model, used CQL (Cassandra Query Language) to perform CRUD operations on Cassandra file system
• Imported data fromCassandrainto HDFS using Mongo export utility.
• Implemented Partitioning, Dynamic Partitions and Buckets 5.5 in HIVE for efficient data access.
• Collaboratedwiththe infrastructure, network, database, application and BI teams to ensuredata qualityand availability.
• Purveyor of competitive intelligence and holistic, timely analyses of BigData made possible by the successful.0
• Implementing various modules in Spark using language scala and Map Reduce processing layer. Able to assess
• Writing Pigscripts for data processing.
• Developed map Reduce jobs to analyze data and provide heuristics reports
• Good experience in writing data ingesters and complex MapReduce jobs in java for data cleaning and preprocessing and fine tuning them as per data sets.
• Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from UNIX, NoSQL and a variety of portfolios.
• Implemented Hive tables and HQL Queries for the reports.
• Involved in data modeling and sharing and replication strategies inMongoDB.
• Worked on hive data warehouse modeling to interface with BI tools such as Tableau.
• Knowledge on handling Hive queries using SparkSQL that integrate Spark environment.
• Exported the analyzed data into relational databases using Sqoop for visualization and to generate reports for the BI team.
• Utilized Agile Scrum Methodology to help manage and organize a team of 4developerswith regular code review sessions.

Environment: ApacheHadoop2x, HDFS, MapReduce 2.0, Hortonworks, Hive1.1.1, Pig0.15.0, HBase1.1.2, Spark0.15.0, Scala, Kafka, Python3.5.1, Sqoop 1.99.6-rc2, Tableaus, Cassandra2.0, Informatica, Oracle 11g/10g, Linux

Confidential - NJ

Hadoop Developer

Responsibilities:

• Analyze large datasets to provide strategic direction to the company.
• Collected the logs from the physical machines and integrated into HDFS using Flume.
• Involved in analyzing the system and business.
• Developed SQL statements to improve back-end communications.
• Loaded unstructured data into Hadoop File System (HDFS).
• Created ETL jobs to load Twitter JSON data and server data into MongoDB and transported MongoDB into the Data Warehouse.
• Created reports and dashboards using structured and unstructured data.
• Involved in importing data from MySQL to HDFS using SQOOP.
• Involved in writing Hive queries to load and process data in Hadoop File System.
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
• Involved in working with Impala for data retrieval process.
• Exported data from Impala to Tableau reporting tool, created dashboards on live connection.
• Sentiment Analysis on reviews of the products on the client's website.
• Exported the resulted sentiment analysis data to Tableau for creating dashboards
• Experienced in Agile processes and delivered quality solutions in regular sprints.
• Developed custom MapReduce programs to extract the required data from the logs.
• Performed performance tuning and troubleshooting of MapReduce jobs by analyzing and reviewing Hadoop log files.
• Responsible for creating Hive tables, loading the structured data resulted from MapReduce jobs into the tables and writing hive queries to further analyze the logs to identify issues and behavioral patterns.
• Responsible for loading and transforming large sets of structured, semi structured, and unstructured data.
• Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and PigScripts. Implementation on Data Loading Part (XML Load)

Environment: Cloudera, JDK 1.5, CDH4.3, Hadoop, Map Reduce, HDFS, Hive, MangoDB, SQOOP, MYSQL, SQL, Impala, Tableau.

Confidential - Portland, OR

Hadoop Admin

Responsibilities:

• Installed and configured Hadoop2.6.0MapReduce2.0, HDFS, Developed multiple MapReduce2.0 jobs in java for data cleaning and preprocessing.
• Worked on Installing and configuring the HDPHorton works 2.x and Cloudera (CDH 5.5.1) Clusters in Dev and Production Environments
• Worked on Capacity planning for the Production Cluster.
• Installed HUE Browser.
• Involved in loading data from UNIX file system to HDFS.
• Involved in creating Hive0.10.0 tables, loading with data and writing hive queries which will run internally in mapreduce way.
• Worked on Installation of HORTONWORKS 2.1 in AZURELinux Servers.
• Worked on Configuring Oozie2.0 Jobs.
• Worked on Configuring High Availability for Name Node in HDP 2.1.
• Worked on Configuring Kerberos Authentication in the cluster.
• Worked on cluster upgradation in Hadoop from HDP 2.1 to HDP 2.3.
• Worked on Configuring queues in capacity scheduler.
• Worked on installing and configuring Solr 5.2.1 in Hadoop cluster.
• Involved in Cluster Monitoring backup, restore and troubleshooting activities.
• Responsible for implementation and ongoing administration of Hadoop infrastructure
• Managed and reviewed Hadoop log files.
• Importing and exporting data from different databases like MySQL, RDBMS into HDFS and HBASE0.94.15 using Sqoop1.4.4.
• Worked on indexing the Hbase tables using Solr and indexing the Json data and Nested data.
• Worked on installation and configuring the Spark2.9 and Impala1.2.3.
• Worked on configuring Queues 5 in Capacity scheduler.
• Worked on configuring queues in Oozie2.0scheduler.
• Worked on Performance Optimization for the Hive queries.
• Worked on Performance tuning in the Cluster level.
• Worked on adding the Users in the clusters.
• Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage & review log files.
• Involved in Analyzing system failures, identifying root causes, and recommended course of actions.
• Creating and deploying a corresponding Solr4.0 Cloud collection.
• Creating collections and configurations, Register a Lily Hbase Indexer configuration with the Lily Hbase Indexer Service.
• Creating and managing the Cron jobs.

Environment: Hadoop2.6.0, MapReduce2.0, Yarn, Hive0.10.0, HDFS, PIG0.12, Sqoop1.4.4, Solr4.0, Oozie2.0, Impala1.2.3, Spark2.9, Cloudera, Flume, HBase, ZooKeeper, CDH5, and Unix/Linux, Hue (Beeswax)

Confidential

Java Developer

Responsibilities:

• Analyzed Business requirements based on the Business Requirement Specification document.
• Involved in System Requirements study and conceptual design.
• Created UML diagrams like activity diagrams, sequence diagrams, and Use case diagrams.
• Developed presentation layer of the project using HTML, JSP 2.0, JSTL and JavaScript technologies.
• Using Micro services based architecture to develop Micro services from a large monolithic.
• Used Object/Relational mapping Hibernate 3.0 framework as the persistence layer for interacting with Oracle 9i.
• Used various Java and J2EEAPIs including XML, Servlets, JSP and JavaBeans.
• Designed and developed Application based on Struts Framework using MVC design pattern.
• Developed Struts Action classes using Struts controller component.
• Written complex SQL queries, stored procedures, functions and triggers in PL/SQL.
• Configured and used Log4j for logging all the debugging and error information.
• Developed Ant build scripts for compiling and building the project. Used SVN for version control of the application.
• Created test plans and JUnit test cases and test suite for testing the application.
• Participated in the production support and maintenance of the project.

Environment: GWT, Java, Web Logic, UNIX OS, CSS, JavaScript, JBoss, AJAX, Eclipse, Perforce, Maven, Hudson, HP Client for Automation, Argo UML, Putty, HP Quality Cente.

Confidential

Java Developer

Responsibilities:

• Designed the application using the J2EE design patterns such as Session Façade, Business Delegate, Service Locator, Value Object and Singleton.
• Developed presentation tier as HTML, JSPs using Struts 1.1 Framework. Used AJAX for faster page rendering.
• Developed the middle tier using EJBs Stateless Session Bean, JavaServlets.
• Entity Beans used for accessing data from the Oracle 8i database.
• Worked on Hibernate for data persistence.
• Prepared high and low level design documents for the business modules for future references and updates.
• Deployed the application in JBoss Application Server in development and production environment.
• Implemented CVS as Versioncontrol system.
• Code Walkthrough/ Test cases and Test Plans.
• Used ANT as build tool. Used Junit for writing Unit tests

Environment: HTML, Java Script, CoreJava, JUnit, JSP, Servlets, JDBC, Oracle 8i, AJAX, CVS and JBoss Application Server.