SUMMARY:

• 8+ years of Professional experience in IT Industry involved in Developing, Implementing, configuring, managing Hadoop ecosystem components and maintenance of various web - based applications using Java, J2EE.
• Experience in installing, configuring and using Hadoop ecosystems such as HDFS, MapReduce, Yarn, Pig, Hive, HBase, Oozie, Zookeeper, Sqoop, Kafka, and Flume.
• Experience in performing Import/export operations from different databases like MySQL, Cassandra, Oracle, Teradata and Netezza into HDFS and vice-versa using Sqoop.
• Good understanding of Classic Hadoop and Yarn architecture with various Hadoop Demons such as Name Node, Secondary Name Node, Data node, Job tracker, Task tracker, Resource Manager, Node Manager, Application Master, and Containers.
• Experience in performing Data transformations, Data cleaning using PIG operations.
• Experience in writing complex Map reduce jobs in Java, Pig Scripts and Hive data modeling.
• Experience in writing PIG and Hive scripts and extending their functionality by writing custom UDF’s and SerDes.
• Experience in handling different file formats like Parquet, Avro, RCFile and Sequence files using Impala.
• Experience in applying Oozie schedulers on Sqoop, Pig, Map Reduce, and Hive.
• Experience in troubleshooting Map Reduce programs and performance tuning of the Hadoop cluster by gathering and analyzing the existing infrastructure.
• Excellent understanding of Zookeeper and Kafka for monitoring and managing Hadoop jobs and used Cloudera CDH 4x, CDH 5x for monitoring and managing Hadoop cluster.
• In-depth understanding of spark architecture including spark core, Spark SQL, Data Frames and Spark Streaming.
• Experience in creating real-time data streaming solutions using Apache Spark/ Spark Streaming / Apache Storm and Kafka
• Migrating the coding from Hive to Apache Spark and Scala using Spark SQL.
• Involved in cluster coordination services through Zookeeper.
• Implemented authorization policies using Sentry for the services such as Hive, Impala, and HDFS access control list.
• Experienced in working with Big Data Hadoop Application using Talend on the cloud through Amazon Web Services (AWS) EC2 and S3.
• Experience in using Business Intelligence tool Tableau and familiar with Data warehousing concepts.
• Experience in creating complex SQL Queries and SQL tuning, writing PL/SQL blocks like stored procedures, Functions, Cursors, Index, triggers, and packages.
• Extensive work experience with different SDLC approaches such as Waterfall and Agile Development Methodologies.

TECHNICAL SKILLS:

Big Data Technologies: Hadoop, Map Reduce, HDFS, Hive, Pig, HBase, Sqoop, Flume, Zookeeper, Oozie, Yarn, Spark, Spark SQL, Apache NiFi

Spark streaming technologies: Spark, Kafka

Programming Languages: C, C++, Java, J2EE, Scala, SQL, PL/SQL and Unix Shell scripting

Frameworks: MVC, Structs, Spring, Junit, and Hibernate.

Developing Tools: Eclipse, Netbeans, IntelliJ Idea, Toad, Maven, and ANT.

Web Languages: XML, HTML,HTML5,DHTML, DOM, JavaScript, AJAX, JQuery, JSON, CSS

Data Bases: Oracle, MySQL, Microsoft SQL Server, Teradata.

No SQL Databases: Cassandra, Janus graph (graph database).

Cloud Computing Tools: Amazon AWS, EC2, S3,EMR.

Business Intelligence Tools: Tableau, Talend(ETL)

Development Methodologies: Agile/Scrum, Waterfall.

Operating Systems & others: Linux (Cent OS, Ubuntu), Unix, Windows XP, Putty, Winscp, FileZilla, Microsoft Office Suite.

PROFESSIONAL EXPERIENCE:

Confidential

Sr. Hadoop Developer

Responsibilities:

• Working on stories related to Ingestion, Transformation, and Publication of data on time.
• Using Spark for real-time data ingestion from web servers (unstructured and structured).
• Implementing data import and export jobs into HDFS and Hive using Sqoop.
• Converting unstructured data into a structured format using Pig.
• Using Hive as a data warehouse in Hadoop, HQL on the data (structured data).
• Using Hive to analyze the partitioned and bucketed using Hive SerDe’s like CSV, REGEX, JSON, and AVRO.
• Using Apache NiFi to check whether the data getting onto Hadoop cluster is a good data without any nulls in it.
• Designing and deploying of Hadoop cluster and different Big Data analytic tools including Pig, Hive, HBase, Oozie, ZooKeeper, Sqoop, Apache Spark and Impala.
• Creating and transformation of RDDs, DataFrame using Spark.
• Working on converting Hive/SQL queries into Spark transformations using Spark RDDs and Scala.
• Working with Big Data Hadoop Application using Talend on cloud through Amazon Web Services (AWS) EC2 and S3. increasing cluster size if needed in AWS using EMR(for data in cloud)
• Developing Spark scripts by using Scala shell commands as per the requirement.
• Using Spark API over Cloudera, Hadoop YARN to perform analytics on data in Hive
• Involving Spark to improvise the performance and optimization of the existing algorithms in Hadoop using Spark context, spark-SQL, Data Frame, pair RDD's, Spark YARN.
• Janus Graph is a Graph database used to store the parent-child relation(ER graph models) between the nodes. The data is stored in Cassandra to understand the nodes and network relations.
• Working with Kafka to get real-time weblogs data onto big data cluster.

Environment: HDFS, Sqoop, Hive, SerDe’s, Hbase, Sentry, Spark, Spark-SQL, Kafka, Flume, Oozie, Json, Avro,Talend,EC2,S3,EMR, Zookeeper, Cloudera.

Confidential, DE

Hadoop Developer

Responsibilities:

• Loading customer data, spending data and credit from legacy warehouses to HDFS
• Exported analyzed data to RDBMS using Sqoop for data visualization.
• Used Hive queries to analyze the large data sets.
• Build reusable Hive UDF’s libraries for business requirements.
• Implemented Dynamic Partitioning and bucketing in Hive.
• Implement script to transmit sys print information from Oracle to HBase using Sqoop
• Deployed the Big Data Hadoop application using Talend on cloud AWS (Amazon Web Service).
• Implemented Map Reduce jobs on XML, JSON, CSV data formats.
• Developed Map reduce programs which were used to extract and transform the data sets and the resultant dataset is loaded to HBase.
• Imported the customers log data into HDFS using Flume.
• Implemented Spark job to improve query performance.
• Experience in designing and architecting data warehouses and data lakes on regular (Oracle, SQL Server) high performance (Netezza and Teradata)
• Used Impala to handle different file formats
• Proactively involved in ongoing maintenance, support, and improvements in Hadoop cluster.
• Used Tableau as a business intelligence tool to visualize the customer information as per the generated records.

Environment: Hadoop, Map Reduce, HDFS, Hive, Sqoop, ZooKeeper, Oozie, Spark, Spark-SQL, Scala, Kafka, Java, Oracle, AWS S3.

Confidential, IL

Big Data Developer.

Responsibilities:

• Worked on Hadoop Cluster with size of 83 Nodes and 896 terabytes capacity
• Developed multiple Map Reduce jobs in Java for data preprocessing.
• Developed Map reduce jobs for analysis according to business requirements.
• Collecting log files, event data and customer behavior (from the devices and apps) and stored in HDFS using Flume.
• Experience in Preprocessing collected data using Pig.
• Maintain, update and schedule the periodic jobs which range from updates on periodic map reduce jobs to creating ad-hoc jobs for the business users.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Written multiple UDF programs in Java for data extraction, transformation, and aggregation from multiple file formats (XML, JSON, and CSV).
• Experience in managing and reviewing Hadoop log files.
• Involved in creating Hive tables, loading with data and writing Hive queries.
• Experience with NO SQL database HBase.
• Scheduled Map Reduce, Pig and Hive jobs using Oozie workflow.
• Experience in running Hadoop streaming jobs to process terabytes of XML format data.
• Created a Data lake by collecting all the information of users activity and profiles which is being used for data analytics.
• The Data Lake is created in Teradata Intellibase.

Environment: Hadoop, HDFS, Pig, Hive, Flume, Sqoop, Hbase, Cassandra, Terradata.

Confidential, Minneapolis

Big Data Developer

Responsibilities:

• Involved as a lead resource to build the entire Hadoop Platform from scratch.
• Responsible for building scalable distributed data solutions using Hadoop.
• Worked on installing the cluster, commissioning & decommissioning of DataNode, NameNode Recovery, capacity planning, and slots configuration.
• Analyze latest Big Data analytics technologies and their innovative applications in both Business intelligence and providing service.
• Loading and transformation of structured, semi-structured and unstructured data to HDFS using Sqoop.
• Achieved increment imports through Sqoop.
• Changed the Replication Factor of files if needed.
• Created a data lake in HDFS collecting all the data needed for organization and collected data from different sources to Data Lake using Flume.
• Extracted the data from the server into HDFS and Bulk loaded the cleaned data into HBase using Map Reduce.
• Used Sqoop to load the previous data from a database to HDFS.
• Written the MapReduce programs, Hive UDFs in java.
• Used Map Reduce JUnit for unit testing.
• Used Maven projects for creating Map Reduce Jar files.
• Developed HiveQL scripts to create, load, and query tables in a Hive Database.
• Created E-mail notification service upon completion of a job for the requested teams.
• Scheduled workflows using Oozie as per the dependencies.
• Played a key role in production after testing the application by BI analysts.
• Maintained system integrity of all sub-components related to Hadoop.
• Implemented Agile methodology to change the Platform.

Environment: Hadoop, Map-Reduce, Java, Maven, Hive, HiveQL, Oozie, HBase, MySQL, Sqoop, Flume, Agile methodology.

Confidential

SQL Admin

Responsibilities:

• Design and developed ER diagram, Normalization and RDBMS concepts.
• Developed SQL Server stored procedures, tuned SQL Queries (using indexes and execution plan)
• Developed UDF’s and Views.
• Created Triggers to maintain the Referential Integrity.
• Designing and Reviewing T-SQL code to verify and meets standards for both performance and error handling according to DBA standards.
• Worked on client requirement and wrote Complex Queries to generate Crystal Reports.
• Tuned and Optimized SQL Queries using Execution Plan and Profiler.
• Rebuilding Indexes and tables as part of Performance Tuning Exercise.
• Working Extensively on Disk Space Utilization and Capacity Planning.
• Involved in performing database Backup and Recovery.

Environment: MS SQL Server, DDL DML, RPO, RTO.

Confidential

Java Developer

Responsibilities:

• Used Collections framework for mapping the categories for the listings.
• Involved in the project which works on Hibernate Spring Framework
• Used Jenkins continuous integration tool to do the deployments.
• Used servlets, javaScript and spring for the server side business logic.
• Developed presentation layer using JSP, HTML, DHTML, CSS, AJAX, JavaScript, and JSTL Taglibs.
• Involved in implementation of an application using conventional design practices (Web-Service Oriented Architecture, MVC (Model View Controller).
• Collaborated on design spec reviews together with Business Analysts.
• Have worked on Oracle 10g database for storing and retrieval of business listings.
• Implemented procedures, packages, triggers, and different Joins to retrieve the database using PL/SQL, SQL scripts. Created DDL, DML scripts to create and retrieve the employee’s data.
• Used version one to work on Agile development.

Environment: JDK 1.6, JSP, HTML,CSS, HTML, AJAX, Java Script, JSTL Taglibs, Oracle 10g, JDBC, MVC architecture, Spring.

Education: Bachelors in Computer Science and Engineering, JNTU University, INDIA.