Big Data Developer Resume

SUMMARY

Having around 3 years of experience in Big data. Which includes around 2 years of professional IT experience as Big data developer and 1 year of freelance projects and tutoring experience.
Knowledge on wide range of Hadoop Components including Hadoop Eco - Systems, Spark, Scala, HDFS, MapReduce, PIG, Hive, and Sqoop in Development and enhancements processes.
Professional Big Data Eco Systems experience in ingestion, storage, querying, processing and analysis of Big Data.
Work involves developing technical design documents, analysis documents, coding to accomplish the requirements, developing and executing Unit Test Plan and preparing Unit Test Results.
Hands on experience in writing Pig & Hive scripts.
Designing and creating Hive external tables using shared meta-store instead of derby with partitioning, dynamic partitioning and buckets.
Extending Hive and Pig core functionality by writing custom UDFs.
Experience in importing and exporting data using Sqoop from Relational Database Systems to HDFS and vice versa.
Excellent understanding and knowledge of NoSQL databases like HBase.
Good knowledge of Oozie concepts like design, development, and execution of workflows in Oozie.
Agility and ability to adapt quickly to changing requirements and scope & priorities
Ability to quickly learn new technologies in a dynamic environment
A good team player and having excellent communication skills with the ability to interact with end users.

TECHNICAL SKILLS

Hadoop/BigData: HDFS, MapReduce, Hive, Pig, HBase, YARN, Spark, Scala, Sqoop, Flume, Zookeeper, Oozie, Kafka

Hadoop Frameworks: Cloudera

AWS: S3

Automation Tool: Jenkins

Languages: SQL, Shell Scripting, Java, Python, R

Database: MySQL, NoSQL, Oracle

Scheduling Tools: Oozie, ESP Scheduler

BI Tools: Tableau

Ticketing Tools: BMC Remedy

IDE: Eclipse, NetBeans, IntelliJ, Visual Studio, BlueJ

PROFESSIONAL EXPERIENCE

Confidential

Big Data Developer

Responsibilities:

Did the cluster setup by using three data nodes with the storage capacity of 1.16TB.
Written the shell script for the flight data for processing in to Hadoop environment Data segregation and summarization was done using Hive and Pig.
Eliminating the duplicates and null values from the data and loading into the hive external tables.
Segregating the data based on Success flightdetails, diverted Flightdetails & delayed flightdetails from the “parenttable” with the yearly dynamic partitions and with joining of carrier description from carrier data, inserting into the corresponding tables.

Confidential

Big Data Developer

Responsibilities:

Written the shell script for the clearance and settlement data for processing in to Hadoop environment and scheduled through gateway servers.
Implemented the Sqoop jobs for importing/exporting data to/from database/HDFS.
Possess SQL and programming knowledge to deal with data transformations and translations.
Monitoring the production jobs using MVS ESP scheduler.
Resolving the abends in a production environment based on the alert mails.
Monitoring the cluster health status using Cloudera Manager (CM).
Providing the support for few of the applications and monitoring the jobs through the ESP scheduler.
Providing the resolution for the failed Hadoop jobs like re-running the jobs, cancelling the jobs or running the jobs from fixlib after changing the code.
Resolving the tickets (Incidents) based on the priority.
Identify information and visualization required to tell a compelling and meaningful story with the various data points available.
Connecting to various data sources including MySQL, Hadoop (Hive/Impala), and Excel.