SUMMARY:

• Having 8+years of overall experience in IT industry which includes implementing, developing and maintenance in Big Data Technologies and Web based applications using Java, J2EE technologies.
• Hands on experience in implementing various solutions and analyzing data using Hadoop Ecosystem like HDFS, MapReduce, Yarn, Spark, Sqoop, Hive, Pig, Flume, Kafka, Impala, Oozie, Zookeeper, NIFI and Cassandra, HBase.
• Hands on experience in various Hadoop distributions like Cloudera (CDH3, CDH4 & CDH5) and Hortonworks Data Platform (HDP).
• Experience in performing in - memory data processing and real time streaming analytics using Apache Spark with Scala , Java and Python .
• Experience in ingestion of data from relational data bases to Hadoop Distributed File System(HDFS), Hive tables and vice versa by using Sqoop tool.
• Experience in Spark Streaming APIs with Kafka to retrieve live streaming data into HBase/Cassandra and optimized it using spark concepts like Data frames, Partitioning, Bucketing, Parallel execution and Map Side Joins, broadcast joins .
• Experience in advanced procedures like text analytics, processing using in-memory computing capabilities like Spark written in Scala .
• Experience in Spark-SQL to process large amount of data by implementing Spark-RDD transformations, Actions and Data frames to required input data.
• Experience on data analysis and developing scripts and queries for pig Latin and Hive QL using Java.
• Experience in using Ambari for provisioning, managing, monitoring and securing apache Hadoop cluster.
• Experienced in handling Avro and Json data in Hive using Hive SerD’s .
• Experience in analyzing data from Cassandra for quick processing, sorting and grouping through CQL.
• Experience on Managing and scheduling Jobs in Hadoop Cluster using Oozie and used Zookeeper for cluster coordination services.
• Good working experience on different file formats like Json, Avro , Parquet , compression techniques like snappy & bzip.
• Hands on experience in data warehousing with ETL tools like Informatica, Ab initio for various loading and transformation processes.
• Good working experience and knowledge in NOSQL databases like HBase, Cassandra and MongoDB .
• Intensive working experience with Amazon Web Services(AWS) using S3 for storage, EC2 for computing and RDS, EBS.
• Experience in NIFI work flow scheduler managing Hadoop jobs by Direct Acyclic graph (DAG) of actions with control flows.
• Designed complete data pipeline from getting logs to loading the metrics related to analytics into reports.
• Experience in using IDEs and source control repositories like Eclipse, IntelliJ and GitHub, Maven, SBT .
• Experience in preparing Tableau reports for analyzed data by using excel sheets, flat files, CSV files.
• Experience in different data sources like Oracle, Netezza, SQL and PostgreSQL.
• Extensive programming experience in Java Core concepts like OOPS , Multithreading and Collections .
• Experience in J2EE technologies like Java servlets, JSP, EJB, and JDBC etc.

TECHNICAL SKILLS:

Hadoop Ecosystem: HDFS, MapReduce, Spark, Hive, Sqoop, Flume, Oozie, Impala, Pig, Kafka, Zookeeper, NIFI and Spark-Streaming, Spark-SQL.

Programming Languages: C, C++, Java, Scala, Python.

Data bases: Oracle, SQL server, MySQL.

NoSQL databases: HBase, Cassandra, Mongo DB.

Operating Systems: Linux, Windows, Ubuntu, Unix.

Source Control and Build Tools: Web servers, Maven, Git, Jenkins and Tableau, Talend.

IDE’s: IntelliJ, Eclipse, NetBeans.

Hadoop Distribution: Cloudera enterprise and Horton works.

Web Technologies: HTML, CSS, JavaScript, jQuery, Ajax, Json.

PROFESSIONAL EXPERIENCE:

Hadoop Developer

Confidential - North Quincy, MA

Roles and Responsibilities:

• Involved in installation, configuration and maintenance of Hadoop clusters for application development with Cloudera distribution.
• Developed end-to-end scalable distributed data pipelines which receiving data using distributed messaging system Kafka through persistence of data into HDFS with Spark-Streaming application using Scala .
• Implemented Spark applications in data processing project to handle data from various sources and creating Data frames on input data which we get from streaming service like Kafka.
• Involved in performance tuning of Spark jobs using Cache and using complete advantage of cluster environment.
• Performed advanced operations like text analytics and processing, using in-memory computing capabilities of Spark using Scala .
• Experience in query data using Spark SQL on Spark to implement Spark RDD’S in Scala.
• Experience in implementing Spark RDD transformations, actions and data frames to required data by using Spark core .
• Developed the Sqoop scripts to make the interaction between Pig and MySQL Database.
• Worked on Partitioning , Bucketing , Parallel execution, Map side Joins for optimization of necessary hive queries.
• Performed Hive QL to create Hive tables and to write Hive queries to perform the data analysis.
• Developed Kafka consumer’s API in Scala for consuming data from Kafka topics .
• Experience in collecting log data from web servers and pushed to HDFS using Kafka and NoSQL database Cassandra.
• Used Pig as ETL tool to do transformations, event joins and some pre-aggregations before storing the data onto HDFS
• Used Oozie workflow to Manage and scheduling Jobs on a Hadoop Cluster and used Zookeeper for cluster coordination services.
• Implemented apache airflow DAG to find popular items in redshift and ingest in the main PostgreSQL via a web service call.
• Generated Java APIs for retrieval and analysis on No-SQL database such as HBase and Cassandra .
• Used NIFI for the transformation of data from different components of Big data ecosystem.
• Automated Cassandra implementation using Chef and OpsWorks in AWS cloud.
• Worked on different data sources like Oracle, Netezza, MySQL, Flat files etc. and experience with AWS components like Amazon Ec2 instances, S3 buckets and Cloud Formation templates.
• Implemented Jira for bug tracking and Bit-bucket to code and code review.
• Used Qlik sense to build customized interactive reports, worksheets, and dashboards.
• Involved in managing and organizing developers with regular code review sessions by utilizing Agile and Scrum Methodologies.

Environment: Hadoop, Hive, Spark, Spark-SQL, Sqoop, Apache Kafka, Cassandra, Scala, Java Oozie, Pig, Eclipse and Qlik sense, Cloudera.

Hadoop Developer

Confidential - St. Louis, MO

Roles and Responsibilities:

• Involved in Collection and aggregation of large amount of weblog data from webservers using Spark Streaming with distributed messaging system Kafka and stored the data into HDFS for analysis.
• Experience on data migrating Map Reduce programs into Spark transformations using Spark with Scala .
• Developed Spark-SQL for implementing Spark RDD transformations, actions, and data frames to generate statistical summary and filtering operations for specific input data.
• Used Spark and Spark-SQL to read the parquet data and create the tables in hive using the Scala API and Spark also used to load JSON data and create Schema-RDD and loaded it into Hive Tables.
• Developed custom mappers in python script and Hive UDFs and UDAFs based on the given requirement.
• Hands on experience on Hive partitioning , bucketing and perform joins on Hive tables and implementing REGEX, JSON and Avro.
• Developed Spark scripts by using Python shell commands as per the requirement.
• Worked on data analytics using R, Spark and Python .
• Writing Pig and Hive scripts with UDF’S in MR and Python to perform ETL on AWS Cloud Services.
• Performed Oozie Operational Services for batch processing and scheduling workflows dynamically.
• Worked on Data Serialization formats for converting Complex objects into sequence bits by using Avro , Parquet , Json , Csv formats.
• Hands on experience on working with Amazon EMR framework transferring data to EC2 server.
• Experience on Amazon Web Services EC2 and S3 by creating buckets, configuring buckets with permissions, logging, tagging and versioning.
• Worked on IntelliJ IDE to developing Scala scripts for Spark jobs.
• Experience in loading data into HBase using bulk load and non-bulk load.
• Knowledge of ETL methods for data extraction, transformation and loading in corporate-wide ETL Solutions and Data warehouse tools for reporting and data analysis.
• Involved in implementation of continues Integration and Deployment framework using Jenkins and Maven.
• Implemented project by using Agile methodologies and attended to Scrum every day.

Environment: Map Reduce, HDFS, Hive, Pig, HBase, SQL, Sqoop, Oozie, Apache Kafka, Eclipse, Spark, Hortonworks, Python and Maven, Git.

Hadoop Developer

Confidential - New Kensington, PA.

Roles and Responsibilities:

• Installed and configured Hadoop Clusters using Cloudera Distribution.
• Experience on Flume to collect, aggregate and store the web log data from web servers, and network devices and stored into HDFS .
• Worked on exporting analyzed data to relational databases using Sqoop to generate the reports for Business intelligence team.
• Implemented Map Reduce jobs using Java and Pig scripts.
• Experience in large-scale Hadoop cluster for distributed data processing and analysis by using Hive and HBase .
• Involved in creating Hive tables by using Impala and working on them using HiveQL and perform data analysis using Hive and Pig .
• Analyzed the large datasets by using Hive queries which invoke and run internally in map reduce program.
• Developed MapReduce jobs for data cleaning and data processing.
• Implemented custom UDF’S, joins and groups for the cleaning and optimization processes on pig scripts.
• Developed MapReduce pipelines jobs to process the data and creating new Hfiles when it is necessary.
• Migrated ETL operations using Pig Latin scripts for joins, filtering, and transformations in to the Hadoop system.
• Involved in loading data from UNIX file system to HDFS using Shell Scripting.
• Used Oozie for job work flow scheduling, managing and monitoring.
• Produced various kinds of reports using Tableau and Power BI based on client requirements.

Environment: Hadoop, MapReduce, Yarn, Hive, HBase, Oozie, Sqoop, Flume, Pig, Linux and Maven, Git.

Java Developer

Confidential

Responsibilities:

• Involved in each phase of Software Development Life Cycle ( SDLC ) models like Requirement gathering and analysis, Design, Implementation, Testing, Deployment and Maintenance.
• Developed Login, Policy and Claims Screens for customers using HTML 5, CSS3, JavaScript , AJAX, JSP, and jQuery .
• Used Core Java to develop Business Logic.
• Involved in the development of business module applications using J2EE technologies like Servlets , JSP .
• Designed and developed the web-tier using JSP's , Servlets framework.
• Used various Core Java concepts such as Multi-Threading, Exception Handling , Collection APIs to implement various features and enhancements.
• Strong experience in design & development of applications using Java/J2EE components such as Java Server Pages (JSP).
• Developed EJB MDB's and message Queue's using JMS technology.
• EJB Session Beans were used to process requests from the user interface and CMP entity beans were used to interact with the persistence layer.
• Developed stored procedures, triggers, and queries using PLSQL in SQL Server.
• Use Spring MVC as framework and JavaScript for client-side view, used frameworks for client-side data validation, creating dynamic web pages-Ajax, jQuery. Developed model classes based on the forms to be displayed on the UI.
• Implemented various design patterns in the project such as Business Delegate, Data Transfer Object, Data Access Object, Service Locator and Singleton.
• Used SQL statements and procedures to fetch the data from the database.
• Developed test cases and performed unit test using JUnit Framework.
• Used CVS as version control and ANT scripts to fetch, build, and deploy application to development environment.

Environments: Java, HTML, CSS, JavaScript, MySQL, Struts, EJB, Spring MVC.

Jr Java Developer

Softech

Responsibilities:

• Performed Analysis, Design, Development, Integration and Testing of application modules.
• Implemented application prototype using JSP , Servlets , JDBC, and to give the presentation.
• Developed new web page designs and development of the project presentation layer by using HTML, JavaScript , JSF , Ajax and implemented CSS for User Interface and better appearance.
• Implemented data base queries by using SQL and PL/SQL to perform data analysis, extraction and various functions.
• Implemented an application by using Struts Framework which leverages classical Model View Layer Architecture ( MVC ).
• Involved in fixing bugs and unit testing with test cases using Junit .
• Worked with various software development methodologies like Agile , Waterfall to increase the development of the project.

Environment: Java, HTML, CSS, JavaScript, JSON, JSP, JDBC and SQL, PL/SQL.