PROFESSIONAL SUMMARY:

• Around 7 years of professional experience in Java/J2EE, Big Data Ecosystem related technologies with full project development, implementation and deployment in various industry vernicles such as Banking, Food & Beverage, Healthcare, Insurance and public sectors.
• 5+ years of valuable industry experience in BigData Analytics with technologies Pig, Hive, HDFS, HBase, Sqoop, Flume, Spark, Kafka, storm, Yarn/MRv2, Oozie, Spark integration with Cassandra, Avro, Shell Scripting Solr and Zookeeper.
• In depth understanding of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, MapReduce concepts.
• Familiar with various Hadoop distributions IBM Big Insights, Cloudera, Horton works and MapR.
• Experienced in writing ad hoc queries using Cloudera Impala.
• Hands on experience in writing Map Reduce jobs in Java, Pig, Hive, Spark.
• Experience with Oozie Workflow Engine in running workflow jobs with actions that run Hadoop MapReduce and Pig jobs.
• Extensively used ETL methodology for supporting Data Extraction, transformations and loading processing, using Hadoop.
• Good understanding of NoSQL databases and hands on work experience in writing applications No SQL Databases HBase, Cassandra and MongoDB.
• Experience in installing and maintaining Cassandra by configuring the Cassandra. yaml file as per the requirement and performed reads and writes using Java JDBC connectivity.
• Hands on experience in Sequence files, RC files, Combiners, Counters, Dynamic Partitions, Bucketing for best practice and performance improvement.
• Extending HIVE and PIG core functionality by using custom User Defined Function's (UDF),User Defined Table - Generating Functions (UDTF) and User Defined Aggregating Functions (UDAF) for Hive and Pig.
• Experience in developing Kafkaconsumers and producers by extending low level and high level consumer and producer API’s.
• Involved in upgrading existing MongoDB instances from version 2.4 to version 2.6 by upgrading the security roles and implementing newer features.
• Expertise in writing Spark Streaming applications using Scala higher order functions.
• Expertise in writing iterative algorithms using Spark MLlib for machine learning applications in Spark-shell on Scala REPL.
• Experience in configuring various topologies in storm to ingest and process data on the fly from multiple sources and aggregate into central repository Hadoop.
• Work experience with cloud infrastructure like Amazon Web Services (AWS).
• Experience in setting up automated monitoring and escalation infrastructure for Hadoop Cluster using Ganglia and Nagios.
• Hands on development experience with JAVA, Shell Scripting, RDBMS, including writing complex SQL queries, PL/SQL, views, stored procedure, triggers, etc.
• Diverse experience in utilizing Java tools in business, Web, and client-server environments including Java Platform, J2EE, EJB, JSP, Java Servlets, Junit, Java database Connectivity (JDBC) technologies and application servers like Web Sphere and WebLogic.
• Familiarity in working with popular frameworks likes Struts, Hibernate, SpringMVC and AJAX and Web Services using XML, HTML and SOAP.

TECHNICAL SKILLS:

Big data/Hadoop Ecosystem: HDFS, MapReduce, Crunch, HIVE, PIG, Sqoop, Flume, Oozie, Spark, Kafka, Storm, HawQ and Avro, Shell Scripting

NoSQL Databases: MongoDB, Cassandra, Hbase

Java / J2EE Technologies: JSE, Servlets, JSP, JDBC,XML,AJAX,SOAP,WSDL

Programming Languages: C, C++, Java, Scala, SQL, PL/SQL, Linux shell scripts.

Database: Oracle 11g/10g, DB2, Vertica, MySQL, TeradataWeb Technologies: HTML, XML, JDBC, JSP, JavaScript, AJAX, SOAP

Frameworks: MVC, Struts 2/1, Hibernate 3, Spring 3/2.5/2.

Tools: Used: Eclipse, IntelliJ, GIT, Putty, Winscp

Monitoring and Reporting tools: Ganglia, Nagios, Custom Shell scripts.

ETL Tools: Informatica, SAP Business Objects, pentaho.

Testing: Hadoop MRUNIT Testing, Hive Testing, Quality Center (QC)

Operating System: Ubuntu (Linux), Win 95/98/2000/XP, RedHat

WORK EXPERIENCE:

Confidential, San Antonio, TX

Hadoop Developer

Responsibilities:

• Extracted and updated the data into HDFS using sqoop import and export command line utility interface.
• Responsible for developing data pipeline using flume, Sqoop and pig to extract the data from weblogs and store in HDFS.
• Involved in using HCATALOG to access Hive table metadata from Map Reduce or Pig code.
• Involved in developing Hive UDFs for the needed functionality.
• Involved in creating Hive tables, loading with data and writing hive queries.
• Managed works including indexing data, tuning relevance, developing custom tokenizers and filters, adding functionality includes playlist, custom sorting and regionalization with Solr Search Engine.
• Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
• Used Pig to do transformations, event joins, filter boot traffic and some pre-aggregations before storing the data onto HDFS
• Used Shell Scripting to move log files into HDFS .
• Experienced in Application Development using Java, J2EE, JSP, Servlets, Struts, RDBMS, Tag Libraries, JDBC, Hibernate, XML and Linux shell scripting.
• Developed data pipeline using Flume, Sqoop, Pig and Java map reduce to ingest customer behavioral data and financial histories into HDFS for analysis.
• Involved in emitting processed data from Hadoop to relational databases or external file systems using SQOOP.
• Data is loaded back to the Sql Server for the BASEL reporting and for the business users to analyze and visualize the data using Datameer.
• Participated in daily scrum meetings and iterative development.
• Loaded all the airline data from the existing DWH tables (SQL Server) to HDFS using Sqoop.
• Orchestrated hundreds of sqoop scripts, pig scripts, hive queries using Oozie workflows and sub-workflows. loaded cache data in to hbase using sqoop. created lots of external tables on hive pointed to hbase tables.
• Analyzed hbase data in hive by creating external partitioned and bucketed tables. worked with cache data stored in Cassandra.
• Used the external tables in impala for data analysis. participated in Apache Spark POCS for analyzing the sales data based on several business factors.

Environment: Hadoop, Map Reduce, HDFS, Pig, Hive, impala, Sqoop, Flume, Oozie, Java, Linux, Maven, Sql Server, Zookeeper, AutoSys, Tableau, Hbase, Cassandra, Apache Spark.

Confidential, Grand Prairie, TX

Sr. Hadoop Developer

Responsibilities:

• Involved in all phases of development activities from requirements collection to production support.
• Migrated from different RDBMS system and focused on migrating from Cloudera distribution to Amazon to reduce project cost.
• Worked with different feeds data like JSON, CSV, XML and implemented data lake concept.
• Understanding the current system and find out the different sources of data.
• Assist in managing, acquiring, and analyzing customer data using SQL and R.
• Predictive analytics (which can monitor inventory levels and ensure product availability).
• Performed Batch processing of logs from various data sources using MapReduce.
• Had an exposure to Amazon Web Services - AWS cloud computing (EMR, EC2 and S3 services).
• Created the Load Balancer on AWS EC2 for unstable cluster.
• As a part of POC, used the Amazon AWS S3 as an underlying file system for the Hadoop and implemented the elastic Map-Reduce jobs on the data in S3 buckets. Backing up HBase data to Amazon S3 at off-peak hours.
• Providing pivotal graphs in order to show the trends.
• Creating Spark SQL queries for faster requests.
• Did POC, Spark integration with Kafka and Cassandra using Java API’s which could be used for customer analytics platform.
• Response to value-added services based on clients' profiles and purchasing habits.
• Build customized memory indexes for high performance information retrieval of products using Apache Lucene and Apache Solr, which provides more precise and useful search data.
• Defined UDFs using PIG and Hive in order to capture customer behavior.
• Design and implement MapReduce jobs to support distributed processing using java, Hive and Apache Pig.
• Create Hive external tables on the MapReduce output before partitioning, bucketing is applied on top of it.
• Reading multiple data formats on HDFS using Scala.
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs and Scala.
• Maintenance of data importing scripts using Hive and MapReduce jobs.
• Worked on hive data warehouse modeling to interface with BI tools including Jaspersoft, Qlikview and Tableau.
• Administered hive permissions & user access with Kerberos authentication.
• Develop and maintain several batch jobs to run automatically depending on business requirements.
• Import and export data between the environments like MySQL, HDFS and deploying into productions.
• Connect tableau from client end with AWS IP addresses and view the end results.
• Developed dashboards, reports, adhoc views, domains in Jasper soft server & Tableau for business/stakeholders.
• Lead project teams on Big Data Analytics using Hadoop framework to analyze massive volumes of unstructured biological data to build a predictive tools which enabled various data scientist to make real time decisions.

Environment: EMR, Hive, PIG, Data Meer, HDFS, Solr, Quartz, Java Map-Reduce, Scala, Spark, Maven, Core Java, GIT, Jenkins, UNIX, R, MYSQL, Eclipse, Oozie, Sqoop, Flume, Jaspersoft, tableau, Qlikview, Cloudera, EMR,EC2,S3,Amazon data pipeline.

Confidential

JAVA/Hadoop Developer

Responsibilities:

• Developed the application using Spring Framework that leverages classical Model View Layer (MVC) architecture UML diagrams like use cases, class diagrams, interaction diagrams, and activity diagrams were used.
• Created Business Logic using Servlets, JSPs, JavaScript, Ajax, Session beans and deployed them on Web logic server.
• Designed the logical and physical data model, generated DDL scripts, and wrote DML scripts for Oracle 9i database.
• Exporting and importing the data from Oracle and copy over to HDFS, Hbase, and Hive using Sqoop .
• Experience in loading logs from multiple sources directly into HDFS using Flume .
• Developed multiple MapReduce jobs in java for data cleaning on unstructured data, pre-processing.
• Performed unit testing using MRUnit .
• The Hive tables created as per requirement were internal or external tables defined with appropriate static and dynamic partitions, intended for efficiency.
• Performed optimization techniques on Mapper Reduce jobs, Hive and Pig queries.
• Experience in developing customized UDF's in java to extend Hive and Pig Latin functionality.
• Implemented partitioning, bucketing in Hive for better organization of the data.
• Analyzed the data using Hive queries and running Pig scripts to study customer behavior.
• Involved in using HCATALOG to access Hive table metadata from Map Reduce or Pig code.
• Performed ETL technique on log data in HDFS using Pig Latin and performed bulk loading data from HDFS to HBase .
• Developed Java client at front-end using thrift server to query the data present in HBase and performed the CRUD Operations on it.
• Experience in designing both time driven and data driven automated workflows using Oozie.
• Provided Technical support for production environments resolving the issues, analyzing the defects, providing and implementing the solution defects.

Environment: JDK 1.5, J2EE 1.4, Springs, JSP, Servlets 2.5, WebSphere 6.1, HTML, XML, ANT 1.6, JavaScript, CDH, HDFS, MapReduce, Pig, Hive, Sqoop, Flume, Hbase, Oozie workflow.

Confidential

Java Developer

Responsibilities:

• Involved in analysis, design and development of the application
• Developed Class Diagrams, Sequence Diagrams, Activity Diagram and Use Cases in Rational Rose.
• Developed the application using Struts MVC 2 architecture.
• Used Struts, Servlets technologies to develop all eligibility Use Cases.
• Worked on loading contracts data from legacy systems to PeopleSoft contracts to create contracts.
• Developed JSP custom tags and Struts tags to support custom User Interfaces.
• Extensively Used Eclipse 3.2 for developing and testing the Security Infrastructure Service.
• Created and maintained the configuration of the Spring Application Frameworks IOC container.
• Developed Web Services for inter organization operations and for retrieving the customer information from the central database using SOAP /HTTP technologies.
• Used Hibernate to store the persistent data into database and written HQL to access the data from the database.
• Designed, developed and maintained the data layer using the ORM framework called Hibernate.
• Coordinated with QA and business team and resolved System Defects generated during testing.
• Experience with Client Side Validations using JSP, and Java script.
• Involved in Design/Code Review with Team and task assignments.
• Installed and Configured BEA WebLogic 9.2.
• Used Spring Framework for Dependency Injection and integrated with Hibernate.
• Used PVCS for version control across common source code used by developers.
• Developed test cases and performed testing using JUNIT.
• Involved design and development of Database in SQL Server 2008.
• Monitored the error logs using Log4J and fixed the problems.

Environment: JDK 1.4/1.5, JNDI, JSP, Hibernate3.x, Struts1.x/2.x, JavaScript, Kronos v.6.1 & v.6.2, Spring, Rational Rose, Eclipse 3.2, Oracle, Unix, Shell, JBoss 4.2, SQL Server 2008, LOG4J, JUNIT, Mockito.