SUMMARY

• Over 5+ years of overall experience in IT as aDeveloper, Designer & Database Administrator with cross platform integration experience usingHadoop and Java/J2EE. Around 2+ years of experience exclusively on Big Data Ecosystem using Hadoop framework and related technologies such as HDFS, HBASE, MapReduce, YARN, Hive, Pig, Flume, Oozie, Sqoop, Spark and Zookeeper.
• Expertise in Big data technologies and Hadoop ecosystem HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce programming paradigm.
• Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop MapReduce, HDFS, HBase, Oozie, Hive, Sqoop, Pig, Zookeeper, Impala, Scala, Storm Flume and YARN.
• Experience in using Cloudera Manager for installation and management of single - node and multi-node Hadoop cluster (CDH3&CDH4). Experience working on NoSQL database (HBase, Cassandra). Experience in Amazon AWS cloud services (EC2, EBS, S3).
• Involved in all phases of Software Development Life Cycle (SDLC) in large scale enterprise software using Object Oriented Analysis and Design.
• Installation of various Hadoop Ecosystems and Hadoop Daemons.
• Hands on experience in installation, configuration, management and deployment of Big Data solutions and the underlying infrastructure of Hadoop Cluster using Cloudera, Hortonworks distributions and MapR.
• Experience in HDFS data storage and support for running map-reduce jobs.
• Used Network Monitoring Daemons like Ganglia and Service monitoring tools like Nagios.
• Decommissioning and commissioning the Node on running Hadoop cluster.
• Backup configuration and Recovery from a Name Node failure.
• Implemented Kerberos security in all environments.
• Manage and review data backups and log files and experience in deploying Java applications on cluster.
• Expertise in Hadoop cluster capacity planning, performance tuning, cluster Monitoring, troubleshooting.
• Experience on Commissioning, Decommissioning, Balancing, and Managing Nodes and tuning server for optimal performance of the cluster.
• Hands on experience in handling structured, semi structured and unstructured data sets.
• Expertise in writing MapReduce jobs using Java code, Pig, Hive for data Processing.
• Experience in developing solutions by analyzing large data sets efficiently.
• Created Hive tables to store data into HDFS and processed data using HiveQL.
• Extending Hive and Pig core functionality by writing custom UDFs.
• Scheduled Oozie workflow engine to run multiple Hive and Pig jobs, which independently run with time and data availability.
• Implemented MapReduce jobs using Java and several MapReduce mechanisms like Combiners, Partitioners, Joins, Data Compressions and Distributed cache.
• Experience in fine-tuning MapReduce jobs for better scalability and performance.
• Expertise in writing Shell-Scripts, Cron Automation and Regular Expressions.
• Good Experience with databases, writing complex queries and stored procedures using SQL and PL/SQL.
• Hands on experience in application development using Java, RDBMS, and Linux shell scripting and writing Queries, Stored procedures, Functions and Triggers by using SQL.
• Expert in developing web page interfaces using JSP, Java Swings, and HTML scripting languages.
• Excellent understanding on Java beans and Hibernate framework to implement model logic to interact with RDBMS databases.
• An excellent team player and self-starter with effective communication skills and proven abilities to finish tasks before target deadlines and highly motivated team player with zeal to learn new technologies.
• Experience in Data architecture including data acquisition, data ingestion, data pipelines, data analytics and analysis. Working knowledge of ETL workflows and pipelines.
• Worked on Import & Export of data using ETL tool Sqoop from MySQL to HDFS.
• Worked on Talend ETL tool for Data Integration.
• Good Knowledge in creating event processing data pipelines using Kafka and Storm.
• Have good experience creating real time data streaming solutions using Apache Spark/Spark Streaming/Apache Storm, Kafka and Flume.
• Worked on importing and exporting data into HDFS and Hive using Sqoop.
• Worked on Talend ETL tool for Data Integration.
• Hands on experience in VPN, Putty, WinSCP, VNC viewer, etc.
• Worked on Continuous Integration and Deployment using GIT, Jenkins, Artifactory.

TECHNICAL SKILLS

Big Data Technologies: Hadoop, HDFS, MapReduce, Pig, Hive, HBase, Impala, Kafka, Sqoop, Zookeeper, Flume, Yarn, Zookeeper, Oozie, Spark, Scala, SolrCloud

Hadoop Technologies and Distributions: Apache Hadoop, Cloudera Hadoop Distribution (HDFS and Map Reduce) - CDH3, CDH4, CDH5 and Hortonworks Data Platform (HDP)

Java Technologies: Java, J2EE, JSTL, JDBC 3.0/2.1, JSP 1.2/1.1, Java Servlets, JMS, JUNIT, Log4j

Frameworks: Struts 1.2, Spring 3.0, Hibernate 3.2

Languages: C, C++, Java, Unix Shell Scripts, Python

Client Technologies: Java Script, CSS, HTML5, XHTML, JQUERY

Databases: Oracle9i/10g/11g, MySQL, SQL/PL SQL

Development Tools: Eclipse, Net Beans, Oracle Developer, Ant, Maven

DB Tools: MySQL, MYSQL developer

ETL Tools: Tableau, Informatica

Operating systems: Windows, Linux, Unix, Ubuntu, Macintosh, CentOS

PROFESSIONAL EXPERIENCE

Confidential

Hadoop Developer

Responsibilities:

• Experience with Cloudera and Horton works distribution of Hadoop.
• Implemented Data Interface to get information of customers using Rest API and Pre-Process data using MapReduce and store into HDFS.
• Imported data from RDBMS to HDFS using Sqoop import/export options.
• Configured Oozie workflows to automate data flow, preprocess and cleaning tasks using Hadoop Actions. Used Oozie for shell actions, java actions and EL.
• Implemented Generic writable to in corporate multiple data sources into reducer to implement recommendation based reports using MapReduce programs.
• Used CGI scripts to access DocDB database.
• Used Cassandra CQL with Java API’s to retrieve data from Cassandra tables.
• Implemented Optimized joins to perform analysis on different data sets using MapReduce programs.
• Implemented HBase features such as compression and use fraud to design, build MapReduce jobs.
• Experienced in optimizing Shuffle and Sort phase in MapReduce Phase.
• Implemented Device based business logic using Hive UDF's to perform ad-hoc queries on structured data.
• Worked extensively with HIVE DDLs and Hive Query language (HQLs).
• Implemented dashboards that internally use Hive queries to perform analytics on Structured data, Avro and Json data to meet business requirements.
• Experienced in handling Avro and Json data in Hive using Hive SerDe's.
• Written Pig scripts to specify the conditions to separate the fraudulent claims.
• Written code with Scala which has support for functional programming.
• Creating Hive tables and working on them using Hive QL. Importing and exporting data into HDFS from Oracle Database and vice versa using Sqoop.
• Implemented test scripts to support test driven development and continuous integration.
• Wrote shell scripts to automate document indexing to SolrCloud in production.
• Experience in managing and reviewing Hadoop log files.
• Used Pig as ETL tool to do transformations, event joins, filter bot traffic and some pre-aggregations before storing the data onto HDFS.
• Worked on custom Pig Loaders and Storage classes to work with a variety of data formats such as JSON, Compressed CSV, etc.
• Used AWS remote computing services such as S3, EC2.

Environment: Hadoop, Hive, MapReduce, HDFS, Pig, Sqoop, Maven, Jenkins, Scala, Java (JDK 1.6), Java 6, Eclipse, Oracle 10g, PL/SQL, SQL*PLUS, Linux.

Confidential, San Francisco, CA

Hadoop Developer

Responsibilities:

• Responsible for building scalable distributed data solutions using Hadoop.
• Gathered the business requirements from the Business Partners and Subject Matter Experts.
• Written multiple MapReduce programs for data extraction, transformation and aggregation from multiple file formats including XML, JSON, CSV & other compressed file formats. Developed Java Map/Reduce job for Trip Calibration, Trip summarization and data filtering.
• Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
• Developed Pig UDFs to provide Pig capabilities for manipulating the data according to Business Requirements and worked on developing custom PIG Loaders.
• Developed Scala scripts, UDF's using both Data frames/Spark SQL and RDD/MapReduce in Spark for Data Aggregation/Transformations, queries and writing data back into RDBMS through Sqoop.
• Expert in implementing advanced procedures like text analytics and processing using the in-memory computing capabilities like Apache Spark written in Scala.
• Worked with different File Formats like TEXTFILE, AVROFILE, ORC and PARQUET for HIVE querying and processing.
• Implemented functionalities using Machine Learning tools like Mahout to display the products best-suited user profiles by performing Sentiment Analysis of the products and performed Trend Analysis of the products and display it to the user.
• Worked on Data loading into Hive for Data Ingestion history and Data content summary.
• Developed Hive UDFs for rating aggregation and HBase java client API for CRUD Operations.
• Created Hive tables and involved in data loading and writing Hive UDFs.
• High sequential throughput of spinning disks was achieved through Kafka.
• Generated Java APIs for retrieval and analysis on No-SQL database such as HBase and Cassandra.
• Worked extensively with Sqoop to move data from DB2 and Teradata to HDFS.
• Collected the logs data from web servers and integrated in to HDFS using Flume.
• Provided ad-hoc queries and data metrics to the Business Users using Hive, Pig.
• Real time analysis along with continuous operation management was processed by Storm.
• Worked on importing and exporting data from Oracle and DB2 into HDFS and HIVE using Sqoop for analysis, visualization and to generate reports.
• Experience in using HBase as backend database for the application development.
• Worked on NoSQL databases including HBase, Cassandra.
• Scheduled Oozie workflow engine to run multiple Hive and Pig jobs, which independently run with time and data availability.
• Involved in running Hadoop Streaming jobs to process Terabytes of data.
• Used JIRA for bug tracking and CVS for version control.

Environment: Hadoop, Spark, MapReduce, Hive, HDFS, PIG, Sqoop, Oozie, Cloudera, Flume, HBase, CDH3, Cassandra, Oracle, J2EE, Oracle/SQL & DB2, Unix/Linux, JavaScript, Ajax, Eclipse IDE, CVS, JIRA.

Confidential, Nashville, TN

Hadoop Admin

Responsibilities:

• Installation of various Hadoop Ecosystems and Hadoop Daemons.
• Used Network Monitoring Daemons like Ganglia and Service monitoring tools like Nagios.
• Importing data from relational data stores to Hadoop using Sqoop.
• Handle the data exchange between HDFS and different Web Apps and DBs using Flume and Sqoop.
• Monitor the data streaming between web sources and HDFS.
• Close monitoring and analysis of the Map Reduce job executions on cluster at task level.
• Worked in Kerberos, Active Directory/LDAP, Unix based File System.
• Experience in Continuous Integration and expertise in Jenkins and Hudson tool.
• Worked on Cloudera test cluster to analyze data present on top of HDFS.
• Worked on large sets of structured, semi-structured and unstructured data.
• Use of Sqoop to import and export data from HDFS to RDBMS and vice-versa.
• Involved in creating Hive tables, loading with data and writing hive queries, which will run internally in MapReduce way.
• Creating various MapReduce jobs for performing ETL transformations on the transactional and application specific data sources.
• Wrote and executed PIG scripts using Grunt shell and big data analysis using Pig and User defined functions (UDF).
• Used HIVE definition to map the output file to tables.
• Setup and benchmarked Hadoop/HBase clusters for internal use.
• Reviewed the HDFS usage and system design for future scalability and fault-tolerance.
• Worked with HBASE NOSQL database.
• Experienced in analyzing Cassandra database and compare it with other open-source NoSQL databases to find which one of them better suites the current requirements.
• Implemented Storm builder topologies to perform cleansing operations before moving data into Cassandra.

Environment: Hadoop, Java 1.4 and Java 1.5, Kerberos, UNIX, Shell Scripting, XML, XSLT, HDFS, HBase, Cassandra, NOSQL, MapReduce, Hive, PIG.

Confidential

Java/J2EE Developer

Responsibilities:

• Involved in the analysis, design, implementation, and testing of the project.
• Implemented the presentation layer with HTML, XHTML and JavaScript.
• Developed web components using JSP, Servlets and JDBC.
• Implemented database using SQL Server.
• Designed tables and indexes.
• Wrote complex SQL and stored procedures.
• Involved in fixing bugs and unit testing with test cases using Junit.
• Study and Analyze 1404 UNIX scripts schedule in crontab to determine the migration plan from the 46 databases schemas, candidate applications shall be analyzed.
• Study and Analyze integration of 1404 jobs in Control M in infodev environment.
• Study and Analyze integration of SFTP implementation for these 1404 jobs.
• Define an approach for identifying the migration path for existing Unix jobs schedule in crontab to Control-M scheduler.
• Perform script migration of all jobs that are part of crontab and migration of selected scripts from FTP to SFTP i.e. secure file transfer protocol.
• A tool and template based approach will be used to perform Migration.

Environment: Java 1.6, UNIX Shell Scripting, MS SQL Server, Eclipse, Putty and Win-SCP.

Confidential

Java/J2EE Developer

Responsibilities:

• Implemented the project according to the Software Development Life Cycle (SDLC).
• Implemented JDBC for mapping an object-oriented domain model to a traditional relational database.
• Created Stored Procedures to manipulate the database and to apply the business logic according to the user’s specifications.
• Developed the Generic Classes, which includes the frequently used functionality, so that it can be reusable.
• Designed and developed user interfaces using JSP, Java script and HTML.
• Involved in Database design and developing SQL Queries, stored procedures on MySQL.
• Used CVS for maintaining the Source Code.
• Logging was done through log4j.

Environment: Java, Java Script, HTML, log4j, JDBC Drivers, Soap Web Services, Unix, Shell scripting, SQL Server.