SUMMARY

• 6+ years of experience in the IT industry, using Java, SQL and Big data technologies in Hadoop ecosystem. Specialize in Design, Development Maintenance of various applications in the Retail/E - Commerce and Telecom.
• Extensive experience working in a dynamic scrum/agile environment.
• Expert in Hadoop ecosystem, including YARN, Oozie MapReduce, Spark, Hive 1.2.2, Sqoop 1.4.6, HBase, Spark 2, Kafka 0.10.2.
• Good knowledge of Hadoop Architecture and its related components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce programming paradigm.
• Have a hands-on experience on fetching the live stream data from web applications using Spark Streaming and messaging services using Apache Kafka.
• Experienced in Developing Spark application using Spark Core, Spark SQL and Spark Streaming API's.
• Good in Spark framework for batch and real-time data processing.
• Good Knowledge of Spark and Scala programming.
• Good Knowledge in apache Nifi.
• Data Ingestion in to Hadoop (HDFS): Ingested data into Hadoop from various data sources like DB2, Oracle, MySQL using Sqoop tool. Created Sqoop job with incremental (delta) load to populate Hive External tables.
• Good in building Map-Reduce Jobs in Java for processing large sets of structured semi-structured and unstructured data sets and stores them in HDFS.
• Expertise experience in writing Hive scripts for processing and analyzing large volumes of structured data.
• Experience in analyzing data using HiveQL.
• Good knowledge in NoSQL databases like HBase and Cassandra.
• Managing and scheduling Jobs to remove the duplicate log data files in HDFS using Oozie.
• Experience in Oozie and workflow scheduler to manage Hadoop jobs by Direct Acyclic Graph (DAG) of actions with control flows.
• Extensive knowledge on file formats like AVRO, Sequence files, Parquet, ORC.
• Experience in managing Hadoop clusters using Cloudera Manager Tool and Ambari.
• Experience in working in Hadoop eco-system integrated to Cloud platform provided byAWS with several services like Amazon EMR instances, S3 bucket.
• Worked with version control systems like GIT for providing common platform for all the developers.
• Experience in working with web services like RESTful and SOAP in JAVA for data transfer.
• Strong programming skills in designing and implementation of applications using Core Java, HTML, WebLogic, J2EE includes JDBC, JSP, Servlets.
• Knowledge of java virtual machines (JVM) and multithreaded processing.
• Used Jira to monitor and track the project updates.
• Good experience in Software development life cycle.
• Followed agile methodology in the project delivery

TECHNICAL SKILLS

Hadoop Technology: \ Programming Languages \Apache Hadoop (MRv2), Hive1.2.2, Sqoop 1.4.6, HBase, FlumeKafka 0.10.2, Spark 2.1.1, Zookeeper, Oozie Java, ScalaSQL/PLSQL.

Databases: \ Web Technologies \Oracle 9g/11g/12c, MySQL 5.7.18, HBase 1.2.6, MS SQL server 2016, Cassandra 3.1.0, Vertica HTML, JSP, CSSIDE’s \ Build Tools \Eclipse, Visual Studio, Micro Strategy Maven, Sbt, Jenkins

Web Services: \ Cloud Technologies \SOAP & RESTful Web Services Amazon Web Services (AWS)

Operating Systems: \ Scripting languages \Windows, Ubuntu Shell

PROFESSIONAL EXPERIENCE

Confidential, Sunnyvale, California

Big Data Engineer

Responsibilities:

• Experience in managing overall system analysis and troubleshooting activities for alarge Big Data analytical solution using batch, delta and streaming flows in Hadoop environment.
• Identify, troubleshoot and resolve production data issues in Hive.
• Used Vertica and postgress to validate the pre-production data.
• Ingested data from various file formats Hadoop - HIVE Warehouse table
• Creating Hive tables on various file formats and working on them using Hive QL.
• Managed and reviewed Hadoop log files.
• Knowledge in job work-flow scheduling and monitoring tools like oozie and Zookeeper.
• Created Hive Dynamic partitions to load time series data
• Responsible to manage data coming from different sources.
• Involved in configuring and managing Hadoop cluster.
• Experienced in working with different data formats like Avro, CSV and Json.
• Used Jenkins CI/CD to trigger the jobs
• Contributed in continuous improvement process for engineering and quality enhancement.
• Graph data processing, storing and creating visual representation of the data using MicroStrategy.
• Created a shell scripts to create schema and upload the data into Hive.
• Followed Agile methodology in the project delivery.
• Used Spark with Scala API’s to analyze the compute the data according to the predefined analytical model in the organization.

Confidential, Beaverton, Oregon

Spark Developer

Responsibilities:

• Used Spark Scala APIs to perform necessary transformations and actions on the data which gets from Kafka in real time.
• Configured Kafka with Spark Streaming to collect the data from the Kafka.
• Experience in handling more than 2 TB of daily data.
• Performed various Parsing technique's using spark API'S to cleanse the data.
• Used Spark API over Horton works Hadoop YARN to perform analytics on data in HDFS.
• Experienced in performance tuning of Spark Applications for setting right Batch Interval time, correct level of Parallelism and memory tuning.
• Analyzed the data using SparkSQL Queries.
• Experienced in working with Spark SQL on different file formats like Avro and Parquet and created data frames for processing.
• Used Spark data frame API for data processing.
• Very good understanding of Partitions, bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance.
• Implemented Incremental Imports of analyzed data into MYSQL tables using Sqoop.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
• Moved Relational Database data using Sqoop into Hive Dynamic partition tables using staging tables.
• Implemented the workflows using Apache Oozie framework to automate tasks.
• Involved in complete Big Data flow of the application starting from data ingestion upstream to HDFS, processing the data in HDFS and analyzing the data.
• Experienced in running Hadoop streaming jobs to process terabytes of xml format data.

Environment: Spark streaming/Scala, Spark SQL, Hive, Kafka, Sqoop, Hortonworks Distribution of Hadoop, Oozie, Java (JDK 1.6), Eclipse, HDFS.

Confidential, Basking ridge, NJ

Hadoop developer

Responsibilities:

• Used Hive and MapReduce to perform the ETL on the data received from the various resource.
• Used Nifi to schedule the data incremental data loads.
• Created Hive queries for extracting data and sending them to clients.
• Very good understanding of Partitions, bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance.
• Performed different types of joins on Hive tables and implementing Hive SerDe.
• Hands on experience in HiveQL.
• Used AWS services like S3 for storage and EMR for data processing.
• Worked on ingesting data from different sources.
• Implemented the workflows using Apache Oozie framework to automate tasks.
• Moved Relational Database data using Sqoop into Hive Dynamic partition tables.
• Exported the analyzed data to relational databases using Sqoop for visualization and to generate reports.
• Followed agile methodology during project delivery.
• Used Git and Jira for project integration and monitoring.

Environment: Hive, Nifi, MapReduce, Sqoop, EMR, S3.

Confidential

Java/J2EE Programmer

Responsibilities:

• Code development and debugging with Eclipse.
• Development of UI using Java Server Pages.
• Developed Java code and Session Beans and Deployed the Application in Web Logic 8.1
• Worked extensively with JSP's and Servlets to accommodate all presentation customizations on the front end.
• Preparation of Unit Test cases.
• Performed Unit Testing, Integration Testing.
• Involved in bug fixing and enhancements

Environment: Core Java, JSP, JDBC, Servlets, ANT, XML, Oracle, DOM, Web Logic, Junit, log4j, Eclipse.

Confidential

Java/J2EE Developer

Responsibilities:

• Worked with functional leads to discuss business specifications and transformed them in to technical requirements.
• Developed environment set up document and assisted development team to install development environment.
• Assisted in developing the technology proposal document required for the system.
• Designed the application workflow using Struts 2.0 and authored struts configuration
• Developed core business components to automate the process of screening applicants for the requirement.
• Developed REST web services for internal consumers.
• Created user interfaces for the presentation layer using JSP, HTML and JavaScript.
• Designed front-end, data driven GUI using JSP, HTML, JavaScript and CSS.
• Developed business components in the rule’s engine using business components. XML based configuration is provided to configure rules engine dynamically on the run time.
• Worked with the database team to analyze the data from development perspective.
• Written JUNIT test cases for unit

Environment: Java, JDBC, Servlets, JSP, Struts 2.0, Web Logic 10.3, Eclipse, Web Services, Oracle 10g, JDBC, HTML, JavaScript, ANT, Soap UI, CVS.

Confidential

Java/J2EE Programmer

Responsibilities:

• Gathered specifications for the Library site from different departments and users of the services.
• Assisted in proposing suitable UML class diagrams for the project.
• Wrote SQL scripts to create and maintain the database, roles, users, tables, views, procedures and triggers in Oracle
• Designed and implemented the UI using HTML, JSP, JavaScript and Java.
• Implemented Multi-threading functionality using Java Threading API
• Extensively worked on IBM Web Sphere 6.0 while implementing the project.
• Involved in Analysis, Design, Coding and Development of custom Interfaces.
• Involved in the feasibility study of the project.
• Gathered requirements from the client for designing the Web Pages.
• Participated in designing the user interface for the application using HTML, DHTML, and Java Server Pages (JSP).
• Involved in writing Client-side Scripts using Java Scripts and Server-Side scripts using Java Beans and used Servlets for handling the business.
• Developed the Form Beans and Data Access Layer classes.
• XML was used to transfer the data between different layers.
• Involved in writing complex sub-queries and used Oracle for generating on-screen reports.
• Worked on database interaction layer for insertions, updating and retrieval operations on data.
• Deployed EJB Components on Web Logic.
• Involved in deploying the application in test environment using Tomcat.

Environment: Java, Servlets, JDBC, HTML, DHTML, SQL Server, Web Logic