SUMMARY

• Around 5 years of professional IT experience with expertise in Java, Oracle, Unix and Big data (Hadoop) ecosystem related technologies.
• 3+years of exclusive experience in Big Data technologies and Hadoop ecosystem components like Spark, MapReduce, Hive, Pig, YARN, HDFS, HBase, Oozie, Sqoop, Zookeeper and Kafka.
• Strong Knowledge of Architecture of Distributed systems and Parallel processing frameworks.
• Strong knowledge on Cloudera distribution and Cloudera components Hue, Cloudera Navigator, Cloudera Manager and Sentry
• In - depth understanding of MapReduce Framework and Spark execution model.
• Strong experience working with both batch and real-time processing using Spark framework.
• knowledge on Cloud - AWS (EC2, EMR, RDS, S3 and DynamoDB).
• Performed Importing and exporting data into HDFS, Hive and HBase using Sqoop.
• Good knowledge on Other distributions like MapR and HDP (Hortonworks Data Platform).
• Involved in Design and Architecting of Big Data solutions using Hadoop Eco System.
• Experience working in large scale Databases like Oracle 11g, DB2,XML, MS Excel and Flat files
• Strong knowledge of performance tuning Hive queries and troubleshoots distinct kinds issues in Hive.
• Good Understandingof custom MapReduce programs &UDF's in Java to extend Hive and Pig core functionality.
• Extensive experience in ETL process consisting of data transformation, data sourcing, mapping, conversion and loading.
• Hands on experience in Capacity planning, monitoring and Performance Tuning of Hadoop Clusters.
• Worked with Sqoop to move (import/export) data from a relational database into Hadoop.
• Knowledge in UNIX Shell Scripting for automating deployments and other routine tasks.
• Experienced in using agile methodologies including SCRUM and Kanban.
• Experience in creating Hive tables with different file formats like Avro, Parquet, ORC.
• Very good understanding of Partitions, bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance.
• Excellent knowledge on Data Analysis, Business Analysis, and Participating in WAR rooms, User story creation for Sprints and Standup meetings.
• Flexible, enthusiastic and project-oriented team player with excellent communication skills with leadership abilities to develop creative solutions for thechallenging requirement of client.

TECHNICAL SKILLS

Big Data Ecosystem: Hadoop, MapReduce, YARN, HDFS, HBase, Zookeeper, Hive, Hue, Pig, Sqoop, Spark, Oozie, Cloudera Manager, Amazon AWS, NiFi, Apache Ambari, Zookeeper, Hortonworks, Impala, Phoenix, Tableau

Languages: Java, PL/SQL, Pig Latin, Python, HiveQL, Scala, SQL

Scripting Languages: Shell Scripting, Java script

Database: Oracle 9i/10g, Microsoft SQL Server, MySQL, DB2, Teradata, PostgreSQL

NOSQL Database: MongoDB,HBase

IDE & Build Tools: Eclipse, Jenkins and Maven.

Version Control System: Subversion, SVN, GIT.

PROFESSIONAL EXPERIENCE

Confidential, Bentonville, AR

Hadoop Developer / Spark Developer

Responsibilities:

• Importing and exporting data into HDFS from MySQL and vice versa using Sqoop. Responsible to manage the data coming from different sources.
• Analyzed requirements and designed data model for Cassandra, Hive from the current relational database in Oracle and Teradata.
• Loaded the customer profiles data, customer spending data, credit from legacy warehouses onto HDFS using Sqoop.
• Analyzed large data sets by running Hivequeries.
• Creating Hivetables, loading with data and writing Hivequeries that will run internally in mapreduce way.
• Created the tasks and detailed the technical documentation for each user story assigned.
• Design, Build, Test, Schedule and Deploy the Oozie workflows.
• Build the Ingestion flows using Oozie for Sqoop jobs to import and export the data
• Designed, Build, Test and deploy Spark SQL programs in SCALA and implement
• Designed, Build, Test and deploy Hive Scripts, Shell Scripts.
• Define and Establish Production Support process Bigdata Platform
• Help the team on Debugging and interact with Cloudera and Infrastructure team for any issues.
• Worked on GIT for version control, JIRA for project tracking and Jenkins for continuous Integration.
• Tuned Hivetable and queries to achieve performance.
• Participated in Release planning, Sprint ceremonies and Daily Stand-upsResponsible for design and development of Big Data applications using ClouderaHadoop.
• Attended the SCRUM meetings and provide the status to Scrum Master and back up to Scrum Master.

Environments: Hadoop 2.x, Pig, HDFS, Scala, Spark, Sqoop, HBase, Oozie, Java, Maven, IntelliJ, GIT, HBase, Putty, Tableau, Map Reduce, HIVE, Teradata, MySql, SVN, Zookeeper, Linux Shell Scripting.

Confidential, Pittsburg, PA

Hadoop Developer

Responsibilities:

• Installed and configured Hadoop Ecosystem components and Cloudera manager using CDH distribution.
• Developed Sqoop scripts to import/export data from Oracle to HDFS and into Hive tables.
• Worked on analyzing Hadoop clusters using Big Data Analytic tools including Map Reduce, Pig and Hive.
• Involved in developing and writing Pig scripts and to store unstructured data into HDFS.
• Involved in creating tables in Hive and writing scripts and queries to load data into Hive tables from HDFS.
• Scripted complex Hive QL queries on Hive tables for analytical functions.
• Optimized the Hive tables utilizing improvement techniques like partitions and bucketing to give better execution Hive QL queries.
• Worked on Hive/Hbase vs RDBMS, imported data to thehive, created internal and external tables, partitions, indexes, views, queries and reports for BI data analysis.
• Used different data formats (Text format and Avro format) while loading the data into HDFS.
• Created tables in HBase and loading data into HBase tables.
• Developed scripts to load data from HBase to Hive Meta store and perform Map Reduce jobs.
• Created custom UDF’s in Pig and Hive.
• Worked Extensively on Cloudera Distribution.
• Involved in exporting data into HDFS from Teradata using Sqoop
• Worked on performance improvement by implementing Dynamic Partitioning and Buckets in Hive and by designing managed and external tables.
• Worked on MapReduce programs to aggregate data from various sources.
• Worked on Sequence files and Avro files in map Reduce programs.
• Created partitioned tables and loaded data using both static partition and dynamic partition methods.
• Installed Oozie workflow engine and scheduled it to run data/time dependent Hive and Pig jobs
• Designed and developed Dashboards for Analytical purposes using Tableau.
• Used Jenkins for mapping the maven and the source tree.
• Analyzed the Hadoop log files using Pig scripts to oversee the errors.

Environment: HDFS, Map Reduce, Hive, Sqoop, Pig, HBase, Oozie, CDH distribution, Java, Eclipse, Shell Scripts, Tableau, Windows, Linux.

Confidential, Boston, MA

Jr. Hadoop Developer

Responsibilities:

• Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hive and Sqoop.
• Developed simple and complex Map Reduce programs in Java for Data Analysis on different data formats
• Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
• Developed Spark program in Scalato analyze reports using Machine Learning models.
• Experienced in implementing static and dynamic partitioning in hive.
• Experience in customizing map reduce framework at different levels like input formats, data types and partitions.
• Extensively Used Sqoop to import/export data between RDBMS and hive tables, incremental imports and created Sqoop jobs for last saved value.
• Involved in loading data from LINUX file system to HDFS.
• Developing Scripts and Autosys Jobs to schedule a bundle (group of coordinators), which consists of variousHadoop Programs using Oozie.
• Created Oozie workflow engine to run multiple Hive jobs.
• Experienced with using different kind of compression techniques to save data and optimize data transfer over network using Snappy in Hive tables.
• Creating Hive tables, loading with data and writing Hive queries which will run internally in Map Reduceway
• Used Zookeeper for providing coordinating services to the cluster.

Environment: Hadoop, Cloudera (CDH 4), HDFS, Hive, Flume, Sqoop, Pig, Java, Eclipse, Teradata, MongoDB, Ubuntu, UNIX, and Maven.

Confidential

ETL / Hadoop Developer

Responsibilities:

• Understanding the Technical design document and Source-To-Target mappings.
• Extensively Used Sqoop to import/export data between RDBMS and hive tables, incremental imports and created Sqoop jobs for last saved value.
• Involved in loading data from LINUX file system to HDFS.
• Developing Scripts and Autosys Jobs to schedule a bundle (group of coordinators), which consists of variousHadoop Programs using Oozie.
• Created Oozie workflow engine to run multiple Hive jobs.
• Creating Hive tables, loading with data and writing Hive queries which will run internally in Map Reduceway
• Used Zookeeper for providing coordinating services to the cluster.
• Designed the ETL process using DataStage tool to load data from source to target Schema.
• Developed various kinds of jobs according to TDD.
• Developing parallel jobs Using stages like Dataset, Sequential file, Aggregator, Copy, Funnel, Lookup, Join, Merge and Transformer Etc.
• Testing the Jobs to assure that data is loaded as per TDD.
• Preparing Unit Test Cases and executing them for each job.
• Moving theETLjobs From Development to production.
• Running, monitoring of the jobs using Data Stage Director and checking logs.

Environment: Informatica Power Center 8.3, Oracle 10g, DB2 9.0, Hadoop, Cloudera, HDFS, Sqoop, PL/SQL, Flat files, MS-SQL Server 2008, MS-Access.