SUMMARY:

• Over 9 years of experience, including 3 years on Big Data Hadoop Eco Systems and 6 years on SQL & PL/SQL, with hands - on project experience in various Verticals which includes financial services and trade compliance
• Expertise in HDFS Architecture & Cluster concepts
• Extensive experience on SQL&PL/SQL, Hadoop and Linux Shell Scripting
• Applied Map Reduce Design Patterns and Writing/Monitoring Map Reduce Programs.
• Expertise in developing Spark applications using Scala
• Migrating the coding from Hive to Apache Spark and Scala using SparkSQL, RDD
• Working experience on Spark, Spark SQL, RDD, Data Frames and Datasets
• Experience in Text normalization and Data cleaning.
• Hands on experience in writing stored procedures and developing crucial interface modules.
• Good Experience in file handling operations on Unix.
• Expertise in ETL operations using Pig Scripting and Pig HBASE integration
• Good Experience in SQL Performance Tuning
• Extensively involved in Requirement analysis, Code development and Bug fixing.
• Expertise in Hive Query Language and debugging Hive issues.
• Expertise in Sqoop and Flume
• Extremely motivated with good inter-personal Skills have ability to work in strict deadlines
• Accomplished facilitator in understanding the client requirements and converting them to the software requirement specifications.

TECHNICAL SKILLS:

Big Data Ecosystems: Hadoop, Map Reduce, HDFS, Hive, Pig, Sqoop, Zookeeper, Oozie, Flume, Spark, SparkSQL, RDD, Data Frames, Datasets, Scala

NOSQL Databases: HBase, Mongo DB, Cassandra

Language: PL/SQL, Oracle, Unix Shell Scripting, Pro*C, HTML

Operating System: MS DOS, MS Windows NT, 2000, XP and HP-UNIX

Database: Oracle 10g, Oracle11g.

Tool: PL/SQL Developer, Toad, Eclipse, Putty.

Version Control tools: PVCS

PROFESSIONAL EXPERIENCE:

Confidential

Hadoop Developer

Environment: Hadoop, MapReduce HDFS, Hive, Pig, Sqoop, Spark, SparkSQL, Scala, Unix, Java

Responsibilities:

• Actively involved in meeting and releasing, working closely with my teammates and managers.
• Written Hive queries for data analysis to meet the business requirements
• Load and transform large sets of structured, semi structured and unstructured data coming from UNIX, NoSQL and a variety of portfolios.
• Migrated data between RDBMS and HDFS/Hive with Sqoop.
• Hands on using Partitions, Bucketing concepts in Hive and designed both Managed and External tables in Hive for optimized performance.
• Used Sqoop to import and export data among HDFS, MySQL database and Hive
• Managing and scheduling jobs on a Hadoop cluster.
• Involved in loading data from LINUX file system to HDFS.
• Experience in Alter the existing Hive tables, loading data and writing Hive queries
• Involved in running Hadoop jobs for processing millions of records of text data.
• Programmed Spark code using Scala for faster processing of data
• Involving in Migrating the coding from Hive to Apache Spark and Scala using SparkSQL, RDD
• Developed Spark with Scala and Spark sql for testing and processing of data

Confidential

Environment: Hadoop, MapReduce HDFS, Hive, Pig, Sqoop, Unix Scripting and JAVA.

Responsibilities:

• Extensively involved in creating Hive tables, and loading and analyzing data using hive queries.
• Loaded data into HDFS and extracted the data from MySQL into HDFS using Sqoop.
• Developed Map Reduce programs, Pig Scripts, Unix shell scripts to parse the raw data, populate staging tables and store the refined data in partitioned tables in the EDW.
• Responsible for requirement gathering & analysis and developing the custom code.
• Enabled speedy reviews and first mover advantages by using Oozie to automate data loading into the Hadoop Distributed File System and PIG to pre-process the data.
• Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from UNIX, NoSQL and a variety of portfolios
• Responsible for managing data coming in from different sources.
• Getting data from ORACLE database using Sqoop.
• Experience in managing and reviewing Hadoop log files.
• Developing Scripts and Batch Job to schedule various Hadoop Program
• Involved in SIT and UAT Support for solving critical issues.

Confidential

Environment: SQL, PL/SQL, and UNIX Shell Scripting.

Responsibilities:

• Created indexes on the tables for faster retrieval of the data to enhance database performance.
• Handled errors using Exception Handling extensively for the ease of debugging and displaying the error messages in the application
• Extensively used collection in PL/SQL objects for improving the performing.
• Modified and rewritten the existing procedures and packages to enhance the performance.
• Analysis of CRs those are raised from UAT & Production.
• Coordinating with the UAT & Production team as well as with the users.
• Extensively used bulk collection in PL/SQL objects for improving the performing.
• Wrote packages to fetch complex data from different tables in remote databases using sub queries.
• Involved in SIT and UAT Support for solving critical issues.
• Involved in requirements, Design phases, Coding, Testing of the functionality.
• Creating and Maintaining Database objects.
• End to end functional testing for entire application.
• Writing and executing the test cases related to developed application.

Confidential

Environment: SQL, PL/SQL, Pro*c, Putty, Sun Solaris.

Responsibilities:

• Extensively involved in writing stored procedures, functions, packages as per the business requirements.
• Redesigned existing procedures and packages to enhance the performance.
• Debugging Pro*C and PL/SQL code block of stored procedures.
• Generation of ad-hoc reports using SQL and stored procedures.
• Involved in the continuous enhancements and fixing of production problems.
• Analysis of CRs those are raised from UAT & Production.
• Coordinating with the UAT & Production team as well as with the users.
• Used Bulk Collections for better performance and easy retrieval of data, by reducing context switching between SQL and PL/SQL engines.
• Wrote SQL, PL/SQL, SQL*Plus programs required to retrieve data using cursors and exception handling
• Involved in SIT and UAT Support for solving critical issues.
• Involved in requirements, Design phases, Coding, Testing of the functionality.
• Creating and Maintaining Database objects.
• End to end functional testing for entire application.

Confidential

Environment: SQL, PL/SQL, Pro*c, Putty, HP-UX 10.

Responsibilities:

• Involved in writing stored procedures, functions, packages as per the business requirements.
• Developed Pro*c programs for flat file generation.
• Worked on Request for Changes (RFC) and Production Problem Resolutions (PPR).
• Provided support across the various phases of the project.
• Prepared and executed unit test cases.