SUMMARY

• Around 6 years of professional IT experience including 2 years in Big data ecosystems, experience in ingestion, storage, querying, processing and analysis of Big Data.
• Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop Map Reduce, HDFS, Hive, Sqoop, Pig, HBase, Oozie and Flume.
• Excellent understanding / knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programming paradigms.
• Good knowledge of Hadoop architecture and various Hadoop Stack elements.
• Extensively used ETL methodology for supporting Data Extraction, transformations and load.
• Experience in analyzing data using HiveQL, Pig Latin, HBase and Map Reduce programs in Java.
• Experience in developing pipelines and processing data from various sources and processing them with Hive and Pig.
• Worked on developing ETL processes (Data Stage & Talend open Studio) to load data from multiple data sources to HDFS using FLUME and SQOOP, and performed structural modifications using Map Reduce, HIVE.
• Extending Hive and Pig core functionality by writing custom UDFs.
• Experience in importing /exporting teh data using Sqoop from HDFS to Relational Database systems and vice - versa.
• Experience in Hadoop administration activities such as installation and configuration of clusters using Cloudera Manager and Apache Ambari.
• Worked on NoSQL databases including HBase, and managing and reviewing Hadoop log files worked with HCatalog, to open up access to Hive’s Metastore.
• Good knowledge in using job scheduling and workflow designing tools like Oozie, Control-M
• Having Good knowledge on YARN Architecture.
• Cutting edge experience onSplunk(Log based performance monitoring tool).
• Expertise in Performance analysis and automation with jMeter and Dynatrace.
• Experience in working with different data sources like Flat files, XML files and Databases.
• Good understanding of Zookeeper and Kafka for monitoring and managing Hadoop Jobs and resources.
• Automated several jobs for pulling data from FTP server and load data into Hive tables, using Oozie workflows.
• Expertise in Unix Commands and automation through shell scripting to process teh flat files.
• Having Good knowledge on Oracle Database Architecture.
• Developed Complex Oracle database objects like Stored Procedures, Functions, Packages and Triggers.
• Extensively used teh Oracle supplied packages, Dynamic SQL, Records and PL/SQL Tables and Object Data Types.

TECHNICAL SKILLS

Big Data Technologies: Hadoop, HDFS, Hive, MapReduce, Pig, Sqoop, Flume, Zookeeper HBase, YARN

RDBMS: Oracle 10g/11g, SQL Server, My SQL

Operating Systems: Linux, Windows 98/00/NT/XP

Languages: JAVA, Unix Shell Scripting, SQL, PL/SQL, C++

Web Technologies, IDE’s& Others: HTML, XML, Eclipse,Control-M,Splunk, Sitescope,TOAD, Putty

Performance Tools: JMeter, Dynatrace

Continuous Integration: Hudson

PROFESSIONAL EXPERIENCE

Confidential, CT

Hadoop Developer

Responsibilities:

• Worked with teh Data Science team to gather requirements for various data mining projects
• Responsible for loading teh raw data sets from file dumps and databases to hdfs.
• Extracted insights from teh datasets in teh format expected by teh data science team
• Written many independent Pig scripts for extracting analytical information.
• Involved in creating Hive tables, and loading and analyzing data using hive queries
• Collected user activity data, log data using Kafka for real time analytics
• Implemented Spark applications from existing MapReduce framework for better performance
• Developed Simple to complex MapReduce Jobs using Hive and Pig.
• Exported teh business required information to RDBMS using Sqoop to make teh data available for BI team to generate reports based on data.
• Involved in running Hadoop jobs for processing millions of records of text data
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required
• Responsible for loading unstructured and semi-structured data into Hadoop cluster from different data sources using Flume
• Developed multiple MapReduce jobs in java for data cleaning and preprocessing
• Responsible for managing data from multiple sources
• Extracted files from CouchDB through Sqoop and placed in HDFS and processed.
• Experienced in runningHadoopstreaming jobs to process terabytes of xml format data.
• Load and transform large sets of structured, semi structured and unstructured data.
• Responsible to manage data coming from different sources.
• Assisted in exporting analyzed data to relational databases using Sqoop
• Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts

Environment: Hadoop, MapReduce, HDFS, Hive, Pig, Java, SQL, Cloudera Manager, Sqoop, Flume, Oozie, Java (jdk 1.6), Eclipse, putty,, PL/SQL, SQL*PLUS, Toad, Putty, Windows NT, UNIX Shell Scripting, Pentaho

Confidential, Ashburn VA

Hadoop Developer

Responsibilities:

• Involved in design and development phases ofSoftware Development Life Cycle (SDLC) usingScrummethodology
• Developed data pipeline using Flume, Sqoop to ingest customer behavioral data and purchase histories into HDFS for analysis.
• Used Pig to perform data validation on teh data ingested using scoop and flume and teh cleansed data set is pushed into Hbase.
• Used Hive to analyze data ingested into Hbase by using Hive-Hbase integration and compute various metrics for reporting on teh dashboard.
• Developed job flows in Oozie to automate teh workflow for pig and hive jobs.
• Loaded teh aggregated data onto Oracle from hadoop environment using Sqoop for reporting on teh dashboard.

Environment: RedHat Linux, HDFS, Map-Reduce, Hive, Java JDK1.6, Pig, Sqoop, Flume, Zookeeper, Oozie, Oracle, HBase.

Confidential

Oracle/Unix Developer

Responsibilities:

• Designed and developed teh interface modules between teh Oracle and Java.
• Generated server side PL/SQL scripts for data manipulation and validation.
• Involved in working with Business to automating many reports using Unix shell scripts on daily basis.
• Involved on working on scheduling jobs in control-m.
• Used Bulk Collections for better performance and easy retrieval of data, by reducing context switching between SQL and PL/SQL engines.
• Extensively used teh advanced features of PL/SQL like Records, Tables, Object types and Dynamic SQL.
• Performed Unit testing for teh Developed modules.
• Involved in teh Requirement Analysis, Done teh Impact analysis To teh Existing system With dis Project.
• Generated server side PL/SQL scripts for data manipulation and validation.
• Involved in Bug Fixes including Job failures and Firecall changes.
• Supporting US, UK and APAC Users.

Environment: Oracle Pl/SQL, Unix Shell scripting, Core Java, Pro C, TOAD, RAPID SQL.Putty, Control-M

Confidential

Oracle/Unix Developer

Responsibilities:

• Designed and developed teh interface modules between teh Oracle and Java.
• Generated server side PL/SQL scripts for data manipulation and validation.
• Performed SQL and PL/SQL tuning and Application tuning using various tools like EXPLAIN PLAN, Hints.
• Extensively involved in using hints to direct teh optimizer to choose an optimum query execution plan.
• Used Bulk Collections for better performance and easy retrieval of data, by reducing context switching between SQL and PL/SQL engines.
• Extensively used teh advanced features of PL/SQL like Records, Tables, Object types and Dynamic SQL.
• Performed Unit testing for teh Developed modules.

Environment: Oracle Pl/SQL, Unix Shell scripting, Core Java, Pro C, TOAD, RAPID SQL.Putty, Control-M, Crystal Reports.