SUMMARY:

• Around 8+ years of experience in IT industry, including Java, SQL, Big data environment, Hadoop ecosystem and Design, Developing, Maintenance of various applications.
• Hands on experience in development, installation, configuring, and using Hadoop & ecosystem components like Hadoop MapReduce, Spark, Scala, HDFS, HBase, Hive, Impala, Sqoop, Pig, Flume, Kafka, Storm, Spark, Elastic Search.
• Excellent understanding of Hadoop architecture and complete understanding of Hadoop daemons and various components such as HDFS, YARN, Resource Manager, Node Manager, Name Node, Data Node HDP and CDH.
• Experience in developing Map Reduce jobs with Java API in Hadoop
• Implemented Data Ingestion using Sqoop into HDFS from RDBMS and vice - versa.
• Involved in developing Pig Latin scripts for Data Transformation and migration
• Handled structured data using Hive.
• Wrote Ad-hoc queries for moving data from HDFS to Hive and analyzed the data using HIVE QL.
• Experience in writing custom UDFs in Java for HIVE and PIG to extend the functionality.
• Good Knowledge on serialization formats like Sequence File, Avro and Parquet Worked with RDBMS including MySQL, Oracle.
• Experience in Cluster maintenance (Adding and removing cluster nodes, Cluster Monitoring and Troubleshooting, Racks, Disk Topology, Manage and review data backups, Manage and review Hadoop log files).
• Loaded local data into HDFS using Apache NiFi.
• Scheduled workflow using Oozie workflow Engine.
• Authentication and authorization management for Hadoop cluster users using Kerberos and Sentry.
• Implementing Map Reduce jobs using Spark, Spark SQL with Scala.
• Experienced with Real-time data processing mechanism in Big Data Ecosystem such as Apache Kafka, Storm, Spark Streaming and Flume
• Experience in data collection, processing and streaming with Kafka.
• Working experience NoSQL Database including HBase and Cassandra.
• Created data visualization with matplotlib, ggplot, GraphX and Tableau for reports.
• Worked with Big Data Hadoop Application using Talend on cloud through Amazon Web Services (AWS) EC2 and S3.
• Experience in working with Teradata. And making the data to be batch processing using distributed computing.
• Good knowledge in Graph databases Janus graph and Neo4j.
• Good hands on experience in NoSQL databases such as HBase, Couldera and MongoDB.
• Front End with HTML5/4, CSS3, JavaScript, jQuery, AngularJS, AJAX, Bootstrap
• Excellent in implementing Object Oriented Programming techniques.
• Strong in core java, data structure, algorithms design, Object-Oriented Design (OOD) and Java components like Collections Framework, Exception handling, I/O system, and Multithreading
• Hands on experience in MVC architecture and Java EE frameworks like Struts2, Spring MVC, and Hibernate.
• Worked in development environment like Git, JIRA, Jenkins, Agile/Scrum and Waterfall

TECHNICAL SKILLS:

Operating System: Windows 2000/NT/ XP/Vista/7 and Linux Red hat, Ubuntu, UNIX.

Languages: C, C++, Scala, Java 1.7/1.6/1.5/1.4/1. X.

Big Data Technologies: HDFS, Hive, Hana, AWS, Map Reduce, Pig, Sqoop, Oozie, Zookeeper, YARN, Avro, Spark, KafkaFrameworks and Utilities: Spring (Spring-Core, Spring AOP, Spring MVC, Spring Batch), Hibernate, Struts.

Databases: Oracle 12c/11g/10g/9i/8i, DB2 UDB, MySQL, MS SQL Server 2000, MS-Access, HBase, MongoDB, Cloudera.

Web technologies: JSP, Servlets, EJB, JNDI, JDBC, Java Beans, HTML, CSS, DHTML, JavaScript, Web Services, SOAP, WSDL, AJAX, jQuery, Angular.JS, Node.JS, Bootstrap, Ext.JS, JSP, JASON, AJAX.

XML technologies: XML, XSL, XSLT, SAX, DOM, AXIS, XML Beans and JAXB.

IDE: RAD 6.x, IBM WSAD 5.1.2, Eclipse, Net beans, JBuilder.

App Server: WebSphere 8.X/7.X/6.X/5.X, WebLogic 7.1/6.1, JBoss, iPlanet.

Web Server: Apache Tomcat 6.0/5.5, Java Web Server 2.0, IIS.

Version Control: Clear case, CVS, RTC, Git.

JUnit, JUnit: Perf, JMock, Cactus, IBM RPT.

Build tool: Ant, Maven, Jenkins

Tools: TOAD, SQL Developer, DB Visualizer, XML SPY, Rational Rose, Server studio., SOAP UI, REST, LOAD UI, Wily, Memory Analyzer.

System Design and Development: Requirement gathering and analysis, design, development, testing, delivery.

PROFESSIONAL EXPERIENCE:

Confidential, Memphis, TN

Sr. Hadoop Developer

Responsibilities:

• Involved in Requirement gathering, Business Analysis and translated business requirements into Technical design in Hadoop and Big Data.
• Worked on Big Data Integration &Analytics based on Hadoop, SOLR, Spark, Kafka, Storm and web Methods.
• Implemented SOA architecture with web services using SOAP, WSDL, UDDI and XML.
• Involved in development of the application using Spring Web MVC and other components of the Spring Framework, the controller being Spring Core (Dispatcher Servlet). Also implemented Dependency Injection using the spring framework.
• Delivered Working Widget Software using EXTJS4, HTML5, RESTFUL Web services, JSON Store, Linux, Hadoop, ZOOKEEPER, NO SQL databases, JAVA, SPRING Security, and JBOSS Application Server for Big Data analytics.
• Implemented the Spring Batch to process large volumes of information that is most efficiently processed without user interaction.
• All the data was loaded from our relational DBs to HIVE using Sqoop. We were getting four flat files from different vendors. These were all in different formats e.g. text, EDI and XML formats
• Involved in migration of data from existing RDBMS (oracle and SQL server) to Hadoop using Sqoop for processing data.
• Written Hive join query to fetch info from multiple tables, written multiple Map Reduce jobs to collect output from Hive
• Worked on migrating MapReduce programs into Spark transformations using Spark and Scala.
• Strongly recommended to bring in Elastic Search and was responsible for installing, configuring and administration.
• Design & implement ETL process using Talend to load data from Worked extensively with Sqoop for importing and exporting the data from HDFS to Relational Database systems/mainframe and vice-versa. Loading data into HDFS.
• Developed data pipeline using Flume, Sqoop, Pig and Java MapReduce to ingest behavioral data into HDFS for analysis.
• Developed Spark jobs using Scala in test environment for faster data processing and used Spark SQL for querying.
• Involved in developing Map-reduce framework, writing queries scheduling map-reduce
• Installed and configured Hadoop and responsible for maintaining cluster and managing and reviewing Hadoop log files.
• Developed a custom AVRO Framework capable of solving small files problem in Hadoop and also extended PIG and Hive tools to work with it.
• Developed Shell, Perl and Python scripts to automate and provide Control flow to Pig scripts.
• Continuous monitoring and managing the Hadoop cluster through Cloudera Manager. \
• Performed Filesystem management and monitoring on Hadoop log files.
• Utilized Oozie workflow to run Pig and Hive Jobs Extracted files from MongoDB through Sqoop and placed in HDFS and processed.
• Implemented Installation and configuration of multi-node cluster on Cloud using Amazon Web Services (AWS) on EC2.
• Used Flume to collect, aggregate, and store the web log data from different sources like web servers, mobile and network devices and pushed to HDFS.
• Implemented partitioning, dynamic partitions and buckets in HIVE.
• Implemented using SCALA and SQL for faster testing and processing of data. Real time streaming the data using with KAFKA.
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
• Involved in Configuring core-site.xml and mapred-site.xml per the multi node cluster environment.
• Used Apache Maven 3.x to build and deploy application to various environments.
• Wrote shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions
• Implemented object/relational persistence (Hibernate) for the domain model and designed and implemented the Hibernate Domain Model for the services.
• Developed and implemented the MVC Architectural Pattern using Struts Framework including JSP, Servlets and Action classes.
• Used parsers like SAX and DOM for parsing xml documents and used XML transformations using XSLT.
• Used ANT automated build scripts to compile and package the application and implemented Log4j for the project.

Environment: Java 1.8, Servlets 3.0, Struts 2.x MVC Framework, Apache Hadoop, HDFS, Hive, Map Reduce, Cloudera, Pig, Sqoop, Kafka, Apache Cassandra, Spark, Scala, Oozie, Impala, Cloudera, Flume, Zookeeper, Hibernate 3, Ant, JDBC, Web Services, IBM WebSphere 7.2, Oracle 11g, Spring Framework 3.1,Spring Batch 2.2, JQuery 1.4, JPA 2.0, JMS, Eclipse Helios 3.6, IBM-RTC, JAX-RPC,JAX-WS,PSE-HSM, Maven, Jenkins, HP - QC, Wily, REST Client, SOAP UI, LOAD UI.

Confidential - New York, NY

Sr. Big data Developer

Responsibilities:

• Evaluated suitability of Hadoop and its ecosystem to the above project and implemented various proof of concept (POC) applications both on Distributed data centers and cloud-based services to eventually adopt them to benefit from the Big Data Hadoop initiative
• Configured Hadoop clusters with Cloudera CDH4 as hot standby to tackle failover situations and to achieve Full Stack Resiliency
• Deployed Hadoop cluster in Azure HDInsight to compare scalability and cost-effectiveness, Queried Hadoop cluster using PowerShell, Hue and as well as the remote console
• Estimated Software & Hardware requirements for the Name-Node and Data-Node& planning the cluster
• Developed ETL transformations that sourced from various sources like e-shopping website and in person shopping records
• Did POC on processing unstructured data in Azure Blob storage
• Integrated Hive and Hbase, loaded data into HDFS and Bulk Loaded the cleaned data into HBase
• Written the Map Reduce programs, Hive UDFs in Java where the functionality is too complex
• Involved in loading data from LINUX file system to HDFS
• Develop HIVE queries for the analysis, to categorize different items
• Designing and creating Hive external tables using shared meta-store instead of the derby with partitioning, dynamic partitioning and buckets
• Given POC of FLUME to handle the real time log processing for attribution reports
• Sentiment Analysis on reviews of the products on the client's website
• Implemented real-time analytics with Apache Kafka and storm
• Tested Spark on real-time data, did frequent item mining on real-time by implementing associative-rule mining
• Exported the resulted sentiment analysis data to Tableau for creating dashboards
• Used Map Reduce JUnit for unit testing
• Maintained System integrity of all sub-components (primarily HDFS, MR, HBase, and Hive)
• Reviewing peer table creation in Hive, data loading and queries
• Monitored System health and logs and respond accordingly to any warning or failure conditions
• Responsible to manage the test data coming from different sources
• Involved in scheduling Oozie workflow engine to run multiple Hive and Pig jobs
• Weekly meetings with technical collaborators and active participation in code review sessions with senior and junior developers
• Created and maintained Technical documentation for launching Hadoop Clusters and for executing Hive queries and Pig Scripts
• Involved unit testing, interface testing, system testing and user acceptance testing of the workflow tool

Environment: Apache Hadoop, HDFS, Hive, Map Reduce, Java, Flume, Horton works, Cloud era, Oozie, My SQL, UNIX, Core Java and Pentaho

Confidential, Phoenix, Arizona

Big Data Engineer

Responsibilities:

• Developed Simple and complex Map Reduce streaming jobs using Java language that have implemented them Using Pig.
• Ingested data into HDFS from Oracle and vice-versa using Sqoop.
• Extensively used Pig for Data cleansing.
• Analyzed the data by running Pig Latin scripts to study customer behavior.
• Handled structured and unstructured data and applying ETL processes.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Developed the Pig UDF'S to pre-process the data for analysis.
• Written multiple UDF programs in Java for data extraction, transformation and aggregation from multiple file formats (XML, JSON, and CSV).
• Develop and maintained complex outbound notification applications that run on custom architectures, using languages Core Java, J2EE, SOAP, XML, JMS, JBoss and Web Services.
• Experienced in monitoring and debug performance issues on Linux (RHEL & Centos).
• Involved in Production Rollout Support which includes monitoring the solution post go-live and resolving any issues that are discovered prior to Rollout.
• Integrating Hadoop with Kafka. Expertise in uploading Click stream data from Kafka to HDFS.
• Designed and documented operational issues by following standards and procedures in a software reporting tool JIRA.

Environment: Pig, Map Reduce, Sqoop, Kafka, Spark, HBase, Oozie, Java, Jiira, Scala, J2EE, XML, SOAP, JSON, JBoss CSV, Linux, RHEL, Centos.

Confidential, Newark, CA

Big Data/Hadoop Developer

Responsibilities:

• Worked o Hadoop Cluster with size of 83 Nodes and 896 terabytes capacity
• Worked on Map reduce jobs, HIVE, Pig.
• Involve in Requirement Analysis, Design, and Development.
• Importing and exporting data into Hive and Hbase using Sqoop from existing SQL server.
• Experience working on processing unstructured data using Pig and Hive.
• Developed Pig Latin scripts to extract data from the web server output files to load into HDFS.
• Implemented Partitioning, Dynamic Partitions, Buckets in Hive.
• Developed Hive queries, Pig scripts, and Spark SQL queries to analyze large datasets.
• Exported the result set from Hive to MySQL using Sqoop.
• Created and maintained technical documentation for launching Hadoop clusters and for executing Hive queries and Pig Scripts.
• Worked on debugging, performance tuning of Hive & Pig Jobs.
• Gained experience in managing and reviewing Hadoop log files.
• Involved in scheduling Oozie workflow engine to run multiple Hive and pig jobs.
• Used NoSQL database with HBase.
• Actively involved in code review and bug fixing for improving the performance.

Environment: Hadoop, HDFS, Pig, Hive, MapReduce, Sqoop, Flume, LINUX, Hbase, Java, Oozie.

Confidential

Software Developer

Responsibilities:

• Interacted with business managers to transform requirements into technical solutions.
• Followed Agile software development with Scrum methodology.
• Involved in Java, J2EE, Struts, Spring, Web Services and Hibernate in a fast-paced development environment.
• Server-side coding and development using Spring, Exception Handling, Java Collections including Set, List, Map, Spring, Hibernate, Webservices, etc in Windows & Linux environment.
• Involved in defect tracking as well as planning using JIRA.
• Resolved a complicate production issue for business managers, where the number of records where displaying wrong.
• Created and modified Struts actions. Worked with struts validations.
• Worked on Spring application framework features IOC container and AOP and integrated Spring with Hibernate using the Hibernate Template.
• Developed enterprise inter-process communication frame work using Spring REST-full Web Service. Developing SOAP Webservices and REST Webservices (JAXB, JSON, JAX-RS, JAX-WS) Developed Hibernate persistent layer.
• Implemented Spring MVC framework in the presentation tier for all the essential control Used Log4j utility to generate run-time logs.
• Prepared Unit and System Testing Specification documents and performed Unit and System testing of the application.
• Reviewed the code for ensuring adherence to java coding standards.
• Developed Functional Requirement Document based on user’s requirement.

Environment: Core java, Servlets, Springs3.0, Spring MVC, Hibernate, REST Web Services, SQL Developer, Apache Tomcat 7.0, MongoDB, Multi-Threading, Web sphere, Agile Methodology, Design Patterns, Apache Maven, Junit.

Confidential

Java Developer

Responsibilities:

• Involved in the complete Software Development Lifecycle (SDLC) using the Agile iterative development
• Methodology. Interacted with the end users and participated in the SCRUM meetings.
• Developing End-User friendly GUI using JSP, HTML, DHTML, JavaScript & CSS
• Implemented CSS Manipulation, HTML Event functions, Java script Effects and animations using JQUERY.
• Involved in development of application using struts, RAD and Oracle database.
• Developed Data Access Layer using Hibernate ORM framework.
• Have coded numerous DAO's using Hibernate Dao Support. Used Criteria, HQL and SQL as the query languages in Hibernate Mapping.
• Used Web Services for transmission of large blocks of XML data using SOAP.
• Used XML for data exchange and schemas (XSDs) for XML validation. Used XSLT for transformation of XML.
• Written numerous test cases for unit testing of the code using JUnit testing framework.
• Used Log4j to implement logging facilities. Used Clear Case for version control.
• Used Ant as a build tool.
• Configured and Deployed application on Web Sphere Application Server.

Environment: Java, Java EE, Web Sphere Application Server, SOAP, Eclipse, Struts, Hibernate, Web Services, HTML, CSS, XML, Ant, UML, JavaScript, jQuery, Rational Rose, JUnit, Log4j, Clear Case, Windows XP.