SUMMARY

• Overall 8+ years of experience in all phases of Software Application requirement analysis, design, development and maintenance of Hadoop/Big Data application and web applications using java/J2EE technologies.Hadoop
• Having 3+ years of hands on experience with Big Data Ecosystems including Hadoop (1.0 and YARN) MapReduce, Spark, Pig, Hive, Sqoop, Flume, Oozie, Zookeeper in a range of industries.
• Good understanding/knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, NameNode, DataNode and MapReduce programming paradigm.
• Experience in writing Hive Queries for processing and analyzing large volumes of data.
• Experience in importing and exporting data using Sqoop from Relational Database Systems to HDFS and vice - versa.
• Developed Oozie workflows by integrating all tasks relating to a project and schedule the jobs as per requirements.
• Automated all the jobs, for pulling data from upstream server to load data into Hive tables, using Oozie workflows.
• Implemented several optimization mechanisms like Combiners, Distributed Cache, Data Compression, and Custom Partitioner to speed up the jobs.
• Used Hbase in accordance with Hive as and when required for real time low latency queries.
• Hands on experience with Spark Core, Spark SQL, Spark Streaming using Scala.
• Used Spark-SQL to perform transformations and actions on data residing in Hive.
• Used Kafka & Spark Streaming for real-time processing.

TECHNICAL SKILLS

Hadoop EcoSystem: Hadoop, Mapreduce, HDFS, Kafka, Hive, Pig, Sqoop, Oozie, Flume, Yarn, Zookeeper, Spark core, Spark Sq.

Programming Languages/: Operating System Java, Scala, SQL, Shell scripting, AngularJS, HTML5, and CSS. Windows, UNIX, Linux distributions (Centos, Ubuntu)

Databases: Cassandra, Hbase, Oracle, DB2, MySQL, SQLite, MS SQL Server .

Development Processes: RUP, AGILE, Scrum

Big data Platforms: Hortonworks, Cloudera.

PROFESSIONAL EXPERIENCE

Confidential, Nashville,TN

Hadoop Developer

Responsibilities:

• Interacted with multiple teams in understanding their business requirements for designing flexible and common component.
• Used Sqoop for importing and exporting data from Netezza, Teradata into HDFS and Hive.
• Worked on avro, parquet file formats by using snappy compression.
• Used Spark Sql with Scala for creating data frames and performed transformations on data frames.
• Implemented Spark SQL to access hive tables into spark for faster processing of data.
• Used Spark API over Cloudera Hadoop YARN to perform analytics on data in Hive.
• Used Spark Sql with Scala for creating data frames and performed transformations on data frames.
• Implemented Spark SQL to access hive tables into spark for faster processing of data.
• Worked on Spark streaming using Apache Kafka for real time data processing.
• Experience in creating Kafka producer and Kafka consumer for Spark streaming.
• Used Hive to do transformations, joins, filter and some pre-aggregations after storing the data to HDFS.
• Automated the jobs with Oozie and scheduled them with Autosys.
• Participated in evaluation and selection of new technologies to support system efficiency.

Environment: Hadoop, Hortonworks, HDFS, Hive, Impala, Spark, Autosys, Kafka, Sqoop, Pig, Java, Scala, Eclipse,, Teradata, UNIX, and Maven.

Confidential, St Louis, Missouri

Hadoop Developer

Responsibilities:

• Developed Spark Programs for Batch processing.
• Worked on Spark SQL and Spark Streaming.
• Developed Spark scripts by using Java, and Scala shell commands as per the requirement.
• Used Spark API over Cloudera Hadoop YARN to perform analytics on data in Hive.
• Used Spark Sql with Scala for creating data frames and performed transformations on data frames.
• Implemented Spark SQL to access hive tables into spark for faster processing of data.
• Installed and configured Hive and also written Hive UDFs.
• Involved in creating Hive tables, loading data and writing Hive queries.
• Imported and exported data into HDFS using Sqoop which includes incremental loading.
• Experienced in defining job flows managing and reviewing Hadoop log files.
• Responsible in managing data coming from different sources.
• Supported MapReduce Programs those are running on the cluster.
• Jobs management using Fair scheduler and Cluster coordination services through Zoo Keeper.
• Involved in loading data from UNIX file system to HDFS.
• Hands on Experience in Oozie Job Scheduling.
• Worked closely with AWS to migrate the entire Data Centers to the cloud using VPC, EC2, S3, EMR.

Environment: Hadoop, MapReduce, HDFS, Pig, Hive, Java, Scala,Spark, Hortonworks, Hbase, Amazon EMR, EC2, S3.

Confidential

Hadoop Developer

Responsibilities:

• Part of team for developing and writing PIG scripts.
• Loaded the data from RDBMS SERVER to Hive using Sqoop.
• Created Hive tables to store the processed results in a tabular format.
• Developed the Sqoop scripts in order to make the interaction between Hive and MySQL Database.
• Developed Java Mapper and Reducer programs for complex business requirements.
• Developed Java custom record reader, partitioner and serialization techniques.
• Created Managed tables and External tables in Hive and loaded data from HDFS.
• Performed complex HiveQL queries on Hive tables and Created custom user defined functions in Hive.
• Optimized the Hive tables using optimization techniques like partitions and bucketing to provide better performance with HiveQL queries.
• Created partitioned tables and loaded data using both static partition and dynamic partition method.
• Performed SQOOP import from Oracle to load the data in HDFS and directly into Hive tables.
• Performed incremental data movement to Hadoop using Sqoop.
• Scheduled mapreduce jobs in production environment using Oozie scheduler.
• Analyzed the Hadoop logs using PIG scripts to oversee the errors caused by the team.
• Experience in gathering requirements from the client, giving estimates for developing projects and delivering the projects in time.

Environment: Java, Hadoop, MapReduce, HDFS, Pig, Hive,Spark, Scala, Hortonworks, Hbase.

Confidential

Java Developer

Responsibilities:

• Involved in Analysis, Design, Development and Testing of the application.
• Incorporated UML diagrams (Class diagrams, Activity diagrams, Sequence diagrams) as part of design documentation and other system documentation.
• Enhanced the Port search functionality by adding a VPN Extension Tab.
• Created end to end functionality for view and edit of VPN Extension details.
• Used AGILE process to develop the application as the process allows faster development as compared to RUP.
• Used Hibernate for persistence framework
• Used Struts MVC framework and WebLogic Application Server in this application.
• Involved in creating DAO’s and used Hibernate for ORM mapping.
• Implemented using Spring Framework for rapid development and ease of maintenance.
• Written procedures, and triggers for validating the consistency of Metadata.
• Written SQL code blocks using cursors for shifting records from various tables based on checks.
• Fixed defects and generated input XML’s to run on SOA Client to generate output XML for testing Web services.
• Written Java classes to test UI and Web services through JUnit and JWebUnit.
• Extensively involved in release/deployment related critical activities.
• Performed functional and integration testing and also tested the entire application using JUnit and JWebUnit.
• Log4J was used to log both User Interface and Domain Level Messages and also used CVS for version control.

Environment: JAVA, JSP, servlets, J2EE, EJB, Struts Framework, JDBC, WebLogic Application Server, Hibernate, Spring Framework, Oracle 9i, Unix, Web Services, CVS, Eclipse, JUnit, JWebUnit.