SUMMARY:

• 8+ years of Professional experience in IT Industry involved in Developing, Implementing, configuring, managing Hadoop ecosystem components and maintenance of various web - based applications using Java, J2EE.
• Experience in installing, configuring and using Hadoop ecosystems such as HDFS, MapReduce, Yarn, Pig, Hive, HBase, Oozie, Zookeeper, Sqoop, Kafka, and Flume.
• Experience in performing Import/export operations from different databases like MySQL, Cassandra, Oracle, Teradata and Netezza into HDFS and vice-versa using Sqoop.
• Good understanding of Classic Hadoop and Yarn architecture with various Hadoop Demons such as Name Node, Secondary Name Node, Data node, Job tracker, Task tracker, Resource Manager, Node Manager, Application Master, and Containers.
• Experience in performing Data transformations, Data cleaning using PIG operations.
• Experience in writing complex Map reduce jobs in Java, Pig Scripts and Hive data modeling.
• Experience in writing PIG and Hive scripts and extending their functionality by writing custom UDF’s and SerDes.
• Experience in handling different file formats like Parquet, Avro, RCFile and Sequence files using Impala.
• Experience in applying Oozie schedulers on Sqoop, Pig, Map Reduce, and Hive.
• Experience in troubleshooting Map Reduce programs and performance tuning of the Hadoop cluster by gathering and analyzing the existing infrastructure.
• Excellent understanding of Zookeeper and Kafka for monitoring and managing Hadoop jobs and used Cloudera CDH 4x, CDH 5x for monitoring and managing Hadoop cluster.
• Experience in installing, configuring, and administrating Hadoop cluster of major Hadoop distributions like Hortonworks and Cloudera.
• In-depth understanding of spark architecture including spark core, Spark SQL, Data Frames and Spark Streaming.
• Experience in creating real-time data streaming solutions using Apache Spark/ Spark Streaming / Apache Storm and Kafka
• Solid programming knowledge on Python.
• Involved in cluster coordination services through Zookeeper.
• Implemented authorization policies using Sentry for the services such as Hive, Impala, and HDFS access control list.
• Experienced in working with Big Data Hadoop Application using Talend on the cloud through Amazon Web Services (AWS) EC2 and S3.
• Experience in using Business Intelligence tool Tableau and familiar with Data warehousing concepts.
• Experience in creating complex SQL Queries and SQL tuning, writing PL/SQL blocks like stored procedures, Functions, Cursors, Index, triggers, and packages.
• Extensive work experience with different SDLC approaches such as Waterfall and Agile Development Methodologies.

TECHNICAL SKILLS:

Big Data Technologies: Hadoop, Map Reduce, HDFS, Hive, Pig, HBase, Sqoop, Flume, Zookeeper, Oozie, Yarn, Spark, Spark SQL, Apache NiFi

Spark streaming technologies: Spark, Kafka, Python

Programming Languages: C, C++, Java, J2EE, Scala, SQL, PL/SQL and Unix Shell scripting

Frameworks: MVC, Structs, Spring, Junit, and Hibernate.

Developing Tools: Eclipse, NetBeans, IntelliJ Idea, Toad, Maven, and ANT.

Web Languages: XML, HTML, HTML5, DHTML, DOM, JavaScript, AJAX, jQuery, JSON, CSS

Data Bases: Oracle, MySQL, Microsoft SQL Server, Teradata.

No SQL Databases: Cassandra, Janus graph (graph database).

Cloud Computing Tools: Amazon AWS, EC2, S3, EMR.

Business Intelligence Tools: Tableau, Talend(ETL)

Development Methodologies: Agile/Scrum, Waterfall.

Operating Systems & others: Linux (Cent OS, Ubuntu), Unix, Windows XP, Putty, Winscp, FileZilla, Microsoft Office Suite.

PROFESSIONAL EXPERIENCE:

Confidential, CA

Hadoop Developer

Responsibilities:

• Working on Unstructured Data, generating daily, weekly, monthly data from the raw fields and storing in the Cluster.
• Running Pig scripts to sort the data for required fields and put back in the cluster.
• Using Python libraries to analyze the data, according to the Business requirement.
• Writing java UDF’s for pig when needed.
• Used Spark to create Data frames.
• Using Splunk to get the analyzed Data.
• Using Numbers to visualize the data.
• Using snappy to compress the files while storing into clusters
• Working in the Cloudera Environment.
• Using Jupyter notebook to run the python code.

Environment: HDFS, Pig, Java, Splunk, Cloudera, Python, Jupyter notebook, Numbers(MAC).

Confidential, SC

Hadoop Developer

Responsibilities:

• Working on stories related to Ingestion, Transformation, and Publication of data on time.
• Using Spark for real-time data ingestion from web servers (unstructured and structured).
• Implementing data import and export jobs into HDFS and Hive using Sqoop.
• Continuous monitoring and managing the Hadoop cluster through Ambari .
• Using Hive as a data warehouse in Hadoop, HQL on the data (structured data).
• Using Hive to analyze the partitioned and bucketed using Hive SerDe’s like CSV, REGEX, JSON, and AVRO.
• Using Apache NiFi to check whether the data getting onto Hadoop cluster is a good data without any nulls in it.
• Using Apache NiFi to filter the error messages in the file and fed back to the data.
• Designing and deploying of Hadoop cluster and different Big Data analytic tools including Pig, Hive, HBase, Oozie, Zookeeper, Sqoop, Apache Spark and Impala.
• Experience on Spark with Scala/Python.
• Working on Admin side, installation and configuration of major Hadoop distributions like Hortonworks and Cloudera.
• Manage several Hadoop clusters in development and production environments.
• Assist development team in identifying the root cause of slow performing jobs / queries (HDFS)
• Working on converting Hive/SQL queries into Spark transformations using Spark RDDs and Scala.
• Working with Big Data Hadoop Application using Talend on cloud through Amazon Web Services (AWS) EC2 and S3. increasing cluster size if needed in AWS using EMR (for data in cloud)
• Developing Spark scripts by using Scala shell commands as per the requirement.
• Using Spark API over Cloudera, Hadoop YARN to perform analytics on data in Hive
• Involving Spark to improvise the performance and optimization of the existing algorithms in Hadoop using Spark context, spark-SQL, Data Frame, pair RDD's, Spark YARN.
• Using PySpark for inserting data from Hive to HBase.
• Janus Graph is a Graph database used to store the parent-child relation (ER graph models) between the nodes. The data is stored in Cassandra to understand the nodes and network relations.
• Working with Kafka to get real-time weblogs data onto big data cluster.

Environment: HDFS, Sqoop, Hive, SerDe’s, Hbase, Sentry, PySpark, Spark-SQL,Python Kafka, Flume, Oozie, Json, Avro,Talend,EC2,S3,EMR, Zookeeper, Cloudera.

Confidential, DE

Hadoop Developer

Responsibilities:

• Loading customer data, spending data and credit from legacy warehouses to HDFS
• Exported analyzed data to RDBMS using Sqoop for data visualization.
• Managing and reviewing Hadoop and HBase log files.
• Used Hive queries to analyze the large data sets.
• Build reusable Hive UDF’s libraries for business requirements.
• Implemented Dynamic Partitioning and bucketing in Hive.
• Implement script to transmit sys print information from Oracle to HBase using Sqoop.
• Deployed the Big Data Hadoop application using Talend on cloud AWS (Amazon Web Service).
• Provisioning, installing, configuring, monitoring, and maintaining HDFS, Yarn, HBase, Flume, Sqoop, Oozie, Pig, Hive and Kafka.
• Implemented Kerberos for authenticating all the services in Hadoop Cluster.
• Implemented Map Reduce jobs on XML, JSON, CSV data formats.
• Developed Map reduce programs which were used to extract and transform the data sets and the resultant dataset is loaded to HBase.
• Imported the customers log data into HDFS using Flume.
• Implemented POC on Apache NiFi a Data flow language.
• Implemented Spark job to improve query performance.
• Experience in designing and architecting data warehouses and data lakes on regular (Oracle, SQL Server) high performance (Netezza and Teradata)
• Used Impala to handle different file formats
• Used Tableau as a business intelligence tool to visualize the customer information as per the generated records.

Environment: Hadoop, Map Reduce, HDFS, Hive, Sqoop, Zookeeper, Oozie, Spark, Spark-SQL, Scala, Kafka, Java, Oracle, AWS S3.

Confidential, IL

Big Data Developer

Responsibilities:

• Worked on Hadoop Cluster with size of 83 Nodes and 896 terabytes capacity
• Developed multiple Map Reduce jobs in Java for data preprocessing.
• Developed Map reduce jobs for analysis according to business requirements.
• Collecting log files, event data and customer behavior (from the devices and apps) and stored in HDFS using Flume.
• Experience in Preprocessing collected data using Pig.
• Maintain, update and schedule the periodic jobs which range from updates on periodic map reduce jobs to creating ad-hoc jobs for the business users.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Written multiple UDF programs in Java for data extraction, transformation, and aggregation from multiple file formats (XML, JSON, and CSV).
• Experience in managing and reviewing Hadoop log files.
• Involved in creating Hive tables, loading with data and writing Hive queries.
• Experience with NO SQL database HBase.
• Scheduled Map Reduce, Pig and Hive jobs using Oozie workflow.
• Experience in running Hadoop streaming jobs to process terabytes of XML format data.
• Created a Data lake by collecting all the information of user’s activity and profiles which is being used for data analytics.
• The Data Lake is created in Teradata Intellibase.

Environment: Hadoop, HDFS, Pig, Hive, Flume, Sqoop, Hbase, Cassandra, Terradata.

Confidential

SQL Admin

Responsibilities:

• Design and developed ER diagram, Normalization and RDBMS concepts.
• Developed SQL Server stored procedures, tuned SQL Queries (using indexes and execution plan)
• Developed UDF’s and Views.
• Created Triggers to maintain the Referential Integrity.
• Designing and Reviewing T-SQL code to verify and meets standards for both performance and error handling according to DBA standards.
• Worked on client requirement and wrote Complex Queries to generate Crystal Reports.
• Tuned and Optimized SQL Queries using Execution Plan and Profiler.
• Rebuilding Indexes and tables as part of Performance Tuning Exercise.
• Working Extensively on Disk Space Utilization and Capacity Planning.
• Involved in performing database Backup and Recovery.

Environment: MS SQL Server, DDL DML, RPO, RTO.

Confidential

Java Developer(Associate Developer)

Responsibilities:

• Used Collections framework for mapping the categories for the listings.
• Involved in the project which works on Hibernate Spring Framework
• Used Jenkins continuous integration tool to do the deployments.
• Used servlets, JavaScript and spring for the server-side business logic.
• Developed presentation layer using JSP, HTML, DHTML, CSS, AJAX, JavaScript, and JSTL Taglibs.
• Involved in implementation of an application using conventional design practices (Web-Service Oriented Architecture, MVC (Model View Controller).
• Collaborated on design spec reviews together with Business Analysts.
• Have worked on Oracle 10g database for storing and retrieval of business listings.
• Implemented procedures, packages, triggers, and different Joins to retrieve the database using PL/SQL, SQL scripts. Created DDL, DML scripts to create and retrieve the employee’s data.
• Used version one to work on Agile development.

Environment: JDK 1.6, JSP, HTML, CSS, HTML, AJAX, Java Script, JSTL Taglibs, Oracle 10g, JDBC, MVC architecture, Spring.