SUMMARY:

• 7+ years’of Professional experience in IT Industry in Developing, Implementing, configuring, testing Hadoop ecosystem components and maintenance of various web based applications using Java, J2EE.
• 3+ years’Real time experience in Hadoop Framework and its ecosystem.
• Experience in installation, configuration and managing - Cloudera’s Hadoop platform along with CDH3&4 clusters.
• Excellent knowledge on Hadoop Architecture and ecosystems such as HDFS, MapReduce, Job Tracker, Task Tracker, Namenode, Datanode andSecondaryNamenodeconcepts.
• Experience in dealing with Apache Hadoop components like HDFS, Map Reduce, Apache Flume, HBase, Hive,Control-M, Apache PIG, Apache Kafka, Apache Solr, Apache Storm and good experience writing Spark core, Spark-SQL and Spark-streaming .
• Good integration experience Apache Spark with Cassandra.
• POC’s on Spark Streaming with various applications inventory data to FCC.
• Experienced with fast and general engine for large-scale data processing in Spark.
• Experience implementing Real Time Stream Processing.
• Experience in using Flume, ApacheKafka to load the log data from multiple sources directly into HDFS in Real Time.
• Experience implementing ApacheStorm topologies - Rich and Trident. Exposure for streaming Event handling using Strom. Having good experience on Storm Topology and writing Spouts and Bolts using Java API.
• Expertise in writing Hadoop Jobs for processing and analyzing data using MapReduce, Hive&Pig. Experienced in extending Hive and Pig corefunctionalitybywriting custom UDFs using Java.
• Hands-on experience on YARN (MapReduce 2.0) architecture and it components.
• Excellent Java development skills using J2EE, spring, J2SE, Servlets, JUnit, JSP, JDBC.
• Hands on experience in application development using Java, UNIX Shell scripting and RDBMS.
• Very good hands-on experience on scripting like AngularJS 1.5.x, Bootstrap 3.3.x.
• Experience configuring and installing and developing scripts on the PRESTO.
• Developed JIL scripts and Scheduled several jobs using Autosys and experienced using GIT and SVN.
• Excellent interpersonal and communication skills, result-oriented with problem solving and leadership skills.
• Excellent conceptual knowledge of NoSQL databases such as HBase, MongoDB, Cassandra.
• Ability to perform at a high level, meet deadlines, adaptable to ever changing priorities.

TECHNICAL SKILLS:

Hadoop/Big Data: HDFS, MapReduce, Hive, Pig, Oozie, Sqoop, HBase, Cassandra, Zookeeper, YARN, TEZ, Flume, Kafka, Storm, Spark, Amazon Web Services (S3, EC2, EMR), Amazon Redshift, Spark-Scala,Presto

Java & J2EE Technologies: Core Java, Servlets, JSP, JDBC, JNDI and Java Beans

Frameworks: MVC, Struts, Hibernate, Spring, AngularJS 1.5.x, Bootstrap 3.3.x

Databases: Teradata, Oracle11g/10g, MySQL, DB2, SQL, NoSQL (Hbase, Cassandra, MangoDB)

Web Technologies: JavaScript, AJAX, HTML, HTML5, CSS3, XML and CSS.

Programming Languages: C, C++, Java, JQuery, Python, UNIX Shell Scripting

IDE: Eclipse, NetBeans

Web Servers: Web Logic, Web Sphere, Apache Tomcat 6

Build Management tools: Maven, Apache ANT, SOAP

Predictive Modelling Tools: Tableau, SAS Editor, SAS Enterprise guide, SAS Miner, IBM Cognos

PROFESSIONAL EXPERIENCE:

Confidential, West Chester, PA

Sr. Hadoop Developer

Responsibilities:

• Interact with business users to consolidate data and processing requirements to support the Enterprise Business Intelligence.
• Involved in DataModeling, System/Data Analysis, Design and Development of data warehouse process models using Hadoop Eco system tools.
• Responsible for building scalable, distributed data solutions using hive, Impala, Spark in Hadoop.
• Created high availability applications without compromising the performance.
• Used ORC File Format and other concepts such as Partitions, Bucketing, Joins, Serde’s and Regex in Hive.
• Exposure to performance tuning and optimization of Hive Queries.
• Moved data from HDFS to RDBMS and vice-versa using SQOOP and TPT.
• Worked on Hive0.14 for implementing Transactional properties such as Updates and Deletes.
• Knowledge on handling Hive queries using SparkSQL that integrate Spark environment.
• Developed Scripts to propagate the data from SOURCE ->PREP ->STG ->JRNL in journalingframework.
• Used NoSQL database Accumulo to store key-value pairs.
• Created scripts to load the date from Hive to Accumulo.
• Developed pig and hivescripts for transforming/formatting the data as per business rules.
• Experienced in Extracting, Transforming, and Loading (ETL) of data from various sources like Flat, Sequence, RC, XML and ORC files, RDBMS and various CompressionTechniques like gzip, bzip2, snappy.
• Automated and scheduled the scripts using AutomicUC4 workflow.
• Worked on AWS to create EC2 instance, configure and maintain Hadoop environment (Zookeeper, Hive and Pig) on those instances.
• Designed and implemented data processing using AWSDataPipeline and worked on cluster coordination services through Zookeeper.
• Installed and configured PRESTO on the cluster and worked on benchmarking testing between PRESTO, Hive and Teradata
• Worked on creating PRESTO views to between the Hive and the Teradata query grid.
• Implemented daily workflow for extraction and processing of data, scheduled jobs using Oozie engine.
• Drafted unit test, integration test plans and created scripts for data warehouse testing, created code reports.
• Interact with team to review designs, code, test plans and complete the project analysis, scoping, scheduling and documentation to ensure quality.
• Proactive participation in the stand up, planning and retrospective meetings which are the life of agile environment to interact with other team member as well as management.

Environment: HortonWorks, Hadoop, MapReduce, Hive, MySQL, Hbase, AWS, HIVE, Impala, PIG, Sqoop, Oozie, PRESTO, Java, DB2, Sell scripting, Solr, JSON, Zookeeper, Eclipse (Kepler), Oracle 11g, PL/SQL, SQL*PLUS, Toad 9.6, UNIX, Tableau, Control-M, MangoDB, Log4j, Kerberos, WinSCP, AWS EC2 cluster.

Confidential, SF, CA

Sr. Hadoop Developer

Responsibilities:

• Designed and implemented Distributed/Cloud Computing (Map Reduce/Hadoop, Pig, Hbase, AVRO, and Zookeeper), AmazonWebServices (S3, EC2, EMR), Installed and configured Hadoop MapReduce, HDFS, developed multiple Map Reduce jobs in java for data cleansing and preprocessing.
• Processed data into HDFS by developing solutions, analyzed the batch data using Map Reduce, Pig, Hive and produce summary results from Hadoop to downstream systems.
• Working with Storm and Hbase to do real time streaming.
• Working on architected solutions that process massive amounts of data on corporate and AWS cloud based servers.
• Setting up the Spark Streaming and Kafka Cluster Developed a SparkStreaming Kafka supplication cluster to Process Hadoop Jobs Logs.
• Designed schema and modeling of data and Written algorithm to store all validated data in Cassandra using Spring Data Cassandra REST
• SparkStreaming App collect user actions data from front end Kafka Producer based Rest API to collect user events and send to Spark Streaming App
• Worked on SparkML library for Recommendations, Coupons Recommendations and Rules Engine.
• Using Scala, Spark& echo system to enriched given data using Fashion Ontology to Validation/Normalizing the data
• MangoDB&Hadoop and Cassandra&Sparkintegrations.
• Involved to writing storm topology for event firing mechanism.
• Migrated from Strom to spark Streaming for event firing using REST calls.
• Using Spark-streaming from web server for analyzing near real time log data.
• Continuous monitoring and managing EMR cluster through AWS Console.
• Working on configuring and Maintaining Hadoop environment on AWS.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Extracted files from RDBMS through Sqoop and placed in HDFS and processed.
• Experience in running Hadoop streaming jobs to process terabytes of xml format data.
• Modifying/creating key spaces and application data tables in Cassandra and Amazon Redshift.
• Worked on various performanceoptimizations like using distributed cache for small datasets, Partition, Bucketing in hive and Map Side joins and Reducer side joins.
• Developed hiveUDF’s using java as and when necessary to use HIVE queries
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
• Replaced default Derby metadata storage system for Hive with MySQL system.
• Developed the PigUDF’s to preprocess the data for analysis.
• Involved in loading data from LINUX and UNIX file system to HDFS.
• Working on managing and transforming the data in to the Amazon Redshift.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig.
• Developed a custom File System plug in for Hadoop so it can access files on Data Platform. This plugin allows Hadoop Map Reduce programs, Hbase, Pig and Hive to work unmodified and access files directly.
• Designed and implemented Map reduce based large scale parallel relation learning system.
• Setup and benchmarked Hadoop/Hbase clusters for internal use.
• Used different file formats like Text files, Sequence Files, Avro, Orc File, and RCF.
• Used various Compression Techniques like gzip, bzip2, snappy, lzo.
• Configuring Zookeeper in order to provide various cluster co-ordination services.
• Used UC4 workflow engine to run multiple Hive and Pig jobs which run independently with time and data availability.
• Involved monitoring Autosys's file watcher jobs and testing data for each transaction and verified data weather it ran properly or not using UC4.
• Very good hands-on experience on scripting like AngularJS 1.5.x, Bootstrap 3.3.x on web front end designing.
• Business Requirement Analysis and Use Case Analysis using RationalRose2000-UML
• Designed the application using Front Controller, ServiceController, MVC, Spring, Factory, Data Access Object, Service Locator and Session Facade Design Patterns.
• Developed classes in RAD6.0 for Java, using the Class Specification provided in Rational Rose.
• Implemented log4j by enabling logging at runtime without modifying the application binary.

Environment: Java (JDK 1.6), Eclipse, Oracle 10g, Hadoop Distribution of Cloudera, Amazon Web Services (S3, EC2, EMR), Hive, Spark, Amazon Redshift, Hbase, Map Reduce, HDFS, Pig, Cassandra,MangoDB, mahout, Oracle 11g/10g, PL/SQL, SQL*PLUS, Toad 9.6, Windows NT, LINUX, UNIX Shell Scripting, Mqseries, AOP, UML/Rational Rose 2000, Log4j, RAD 6.0, Tomcat 4.0, Rational ClearCase, Sun/Solaris, AngularJS 1.5.x, Bootstrap 3.3.x, HTML5, CSS3.

Confidential, NYC, NY

Hadoop Developer

Responsibilities:

• Installed and configured Hadoop MapReduce, HDFS, Developed multiple MapReduce jobs in Java for data cleansing and processing.
• Used Spark API over Hortonworks Hadoop YARN to perform analytics on data in Hive.
• Exploring with the Spark improving the performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frame, Pair RDDs, Spark YARN.
• Load the data into Spark RDD and do in memory data Computation to generate the Output response.
• Designed and implemented Distributed/Cloud Computing (Map Reduce/Hadoop, Pig, Hbase, AVRO, and Zookeeper), AmazonWebServices.
• Written MapReduce code to process and parsing the data from various sources and storing the parsed data into HBase and Hive using Hive integration.
• Experience on continuous monitoring and managing EC2 cluster through AWS Console.
• Working on configuring and Maintaining Hadoop environment on AWS.
• Worked with Hbase and Hive scripts to extract, transform and load the data into HBase and Hive.
• Designed appropriate partitioning/bucketing schema to allow faster data retrieval during analysis using HIVE.
• Extensive working knowledge of partitioned table, UDFs, performance tuning, compression related properties in Hive.
• Worked with different File Formats like TEXTFILE, AVROFILE, and ORC for HIVE querying and processing.
• Used various Compression Techniques like gzip, bzip2, snappy, lzo.
• Worked on installing cluster, commissioning & decommissioning of Datanode, Namenode recovery, capacity planning and slots configuration.
• Designed schema and modeling of data in Cassandra
• Used Cassandra CQL with Java API's to retrieve data from Cassandra tables
• Written algorithm to store all validated data in Cassandra using Spring Data Cassandra REST
• Cassandra&Hadoop and Cassandra&Spark integrations.
• Implemented Flume to collect data from various sources and is loaded into HDFS for further processing.
• Developed workflows using custom MapReduce, Pig, Hive and Sqoop.
• Good working experience using Sqoop to import data into from RDBMS.
• Tuned the cluster for optional performance to process these large datasets.
• Built reusable Hive UDFs to sort structure fields and return complex datatype.
• Responsible for loading data from UNIX file system to HDFS.
• Developed suit of Unit Test Cases for Mapper, Reducer and Driver classes using MR testing library.
• Developed workflow in Control-M to automate tasks of loading data into HDFS, preprocessing with PIG.
• Used Maven extensively for building jar files of MapReduce programs and deployed to cluster.
• Used OOZIE engine for creating workflow jobs for executing Hadoop jobs such as Hiveand Sqoop operations.
• Modelled Hive Partitions extensively for data separation and faster data processing and followed Pig and Hive best practices for tuning.
• Partitioned each day’s data into separate partitions for easy access and efficiency.
• Used Log4J for standard logging and used JacksonAPI for JSON request handling.
• Extensively worked on Shell Scripts.
• Performing testing in various environments (Dev, ANA, PREF) and providing the reports to the business.
• Exported the analyzed data to relational databases using Sqoop for visualization and to generate reports.
• Integrated BI tool with Impala.
• Cluster coordination services through Zookeeper.
• Also assisted admin team in installed and configuration of additional nodes in Hadoop cluster
• Used Visualization tools such as Power view for excel, Tableau for visualizing and generating reports.
• Used GIT and ClearCase for software configuration management in collaboration with clearquest to log and keep track of SCM Activities.

Environment: CDH5.3.2, Hadoop, MapReduce, Hive, MySQL, Hbase, AWS, HIVE, Impala, PIG, Sqoop, Oozie, Flume, Java, DB2, Sell scripting, Solr, JSON, Zookeeper, Eclipse (Kepler), Oracle 11g, PL/SQL, SQL*PLUS, Toad 9.6, UNIX, Tableau, Control-M, MangoDB, Log4j, Kerberos, WinSCP, AWS EC2 cluster.

Confidential, Concord, NC

Hadoop Developer

Responsibilities:

• Interacted daily with the onshore counterparts to gather requirements.
• Worked on few development projects on Teradata and gained a knowledge on Teradata utilities like Mload, Tpump, Bteq etc.
• Performed Input data analysis, generated space estimation reports for Staging and Target tables in Testing and Production environments.
• Worked on a Remediation project to optimize all the Teradata SQL queries in e-commerce line of business and applied several query tuning and query optimization techniques in Teradata SQL.
• Converted projects from Teradata to Hadoop as Teradata proved to be costly to handle huge data of the Bank, thus gained knowledge on Hadoop right from scratch.
• Installed and configured Hadoop MapReduce, HDFS and started to load data into HDFS instead of Teradata and performed Data Cleansing and Processing operations.
• Have gained an in-depth knowledge on HDFS data storage and MapReduce data processing techniques.
• Performed importing and exporting data into HDFS and Hive using Sqoop.
• Designed and Scheduled workflows using Oozie.
• Created UNIX Shell Scripts to within turn call several Hive and Oozie scripts.
• Built several Managed and External HIVE tables and performed several Joins on those tables to achieve the result.
• Modelled Hive Partitions extensively for data separation and faster data processing and followed Pig and Hive best practices for tuning. Partitioned each day’s data into separate partitions for easy access and efficiency.
• Scheduled several jobs which include several Shell, Oozie, hive and SQOOP scripts using Autosys. Coded JIL scripts to determine the job dependencies while scheduling.
• Performed Data analytics on the credit card transactional data and Coded Automatic Report Mailing Scripts in UNIX.
• Worked in Testing and Production environments and learnt moving components from one environment to the other Subversion (SVN).
• Performed Encryption and Decryption of key fields like Account Number of the Bank customers in several input and reporting files using COBOL.
• Monitor Hadoop jobs on Performance and Production cluster. Provided Production support for few successful runs.
• Built complete ETL logic, generated transformations, work flows and automated the scheduled runs.
• Built an Automatic Query Performance Metrics Generation Tool in Teradata using SQL Procedures.
• Generated Test scripts and Test plan, Data Analysis and Defect Reporting using HP Quality Center.

Environment: Hadoop, Teradata, HDFS, MapReduce, HIVE, PIG, Sqoop, Oozie, Java, COBOL, Autosys

Confidential, Dover, NH

Java Developer

Responsibilities:

• Designed the project using MVC, Session Façade, and service Locator and Business Delegate patterns.
• Followed agile methodology, interacted with the client, provide/take feedback on the features, suggest/ implement optimal solutions and tailored application to customer needs.
• Integrated the application with Spring Framework for implementing Dependency Injection and provide abstraction between presentation layer and persistence layer.
• Developed Web pages using Struts view component JSP, JavaScript, HTML, jQuery, AJAX, to create the user interface views Migration 3rd party applications.
• Implemented validation on the JSP fields using Struts Validation framework which included writing validation.xml and validation-rules.xml files.
• Implemented all DAO’s using Spring Framework. Implemented SOA using Web Services (SOA over WESB and SOAP over JMS) on ESB utilizing WebSphere Integration Developer(WID).
• Used Hibernate ORM framework as persistence engine, configured O/R mapping and wrote hibernate queries.
• Involved in designing front-end screens using Java Script, JSP, Angular JS, AJAX, HTML5 and DHTML.
• Designed Presentation layer using JSP, Struts framework and did front-end validations using JavaScript and jQuery.
• Involved in design and development of UI component, using frameworks Angular JS, JavaScript, HTML5, CSS and Bootstrap.
• Followed Single Page Application(SPA) Pattern and used Angular JS MVC to build a cross-browser complaint application.
• Implemented core features of Angular JS framework such as dependency Injection, Data-binding, Filters, Directives, Templates, Services and deep linking.
• Designed and integrated the full scale Hibernate/Spring.
• Developed hibernate DAO framework with spring framework.
• Configured and deployed the application on WebLogic Application Server.
• Implemented Complex Business Delegate layer using DAO’s created and business logic.
• Exposed various capabilities such as Web Services using SOAP and WSDL.
• Worked on J2EE components like Servlets, JSPs and MDBs.
• Used MVC architecture, Business Delegate, Session façade, Data Access Object, Value Object and Singleton patterns.
• Used Xfire to read the XML response coming from the web service.
• Created Junit test cases and Development of Junit classes.
• Configured Log4j to enable/disable logging in application.
• Developed ANT script to build .war and .ear files for the application.

Environment: Java, Eclipse, J2EE1.3, Struts 1.0.1/1.0.2, UML/Rational Rose 2000, Log4j, JMS, JSP, XML, XSL, EJB, Servlets, JBoss3.x, Oracle 8I, Rational ClearCase, Mainframes, Sun/Solaris, Oracle 8.x/9i, Toad, CTG, Cobol.

Confidential, Dublin, OH

Jr. Java Developer

Responsibilities:

• Developed the user interface screens using web technologies like JSP, JavaScript, HTML, AJAX, CSS,and XSLT.
• Modeling conceptual design using Use Case, UML Class and Activity diagrams using Rational Rose.
• Wrote requirement specific SQL and PL/SQL scripts including Stored Procedures, functions, packages and triggers.
• Implemented Database access through JDBC at Server end with Oracle.
• Used Spring Aspect Oriented Programming (AOP) for addressing cross cutting concerns.
• Developed request/response paradigm by using Spring Controllers, Inversion of Control and Dependency Injection with Spring MVC.
• Used Web Services like SOAP and WSDL to communicate over internet.
• Involved in implementation of the JMS Connection Pool, including publish and subscribe using Spring JMS.
• Used CVS for version control and Log4j for logging.
• Used JProbe and JConsole to profile application for memory leaks and resource utilization.
• Developed test classes in JUnit for implementing unit testing.
• Deployed the application using WebLogic Application Server.
• Made enhancements to the application which presented me with the opportunity to go through the entire SDLC.
• Providing daily updates to the on-site team over call and making enhancements.
• Assisted the Quality Assurance team in testing the applications.
• Code coverage and Test case presentation.

Environment: Java, JDK1.4, Eclipse Juno, J2EE, JDBC, Servlets, JSP, JSTL, HTML, AJAX, Spring, Java Script, CSS, XSLT, XML. JUnit, WebServices, SOAP, WSDL, REST, Maven, Json, Weblogic, CVS, Rational Application Developer RAD, Hibernate, Rational Rose, JMS.