Hadoop Developer Resume

SUMMARY

Masters of Electrical & Computer Engineering with Two years of Hadoop Developer experience
Experience on Hadoop Ecosystems projects such as Map - Reduce, HIVE, Spark.
Experience in processing data using HIVEQL and Pig Latin and custom Map Reduce programs in Java and shell script.
Used Spark with Kafka to implement a receiver and process data using spark streaming.
Used Sqoop to transfer bulk data from Relational databases to Hadoop and vice versa.
Collecting and aggregating large amount of Log Data using Apache Flume and storing data in HDFS for further analysis.
Worked at Job/Workflow scheduling and monitoring tools like Oozie and Zookeeper.
Experience in different phases of Software Development Life Cycle (SDLC) including Design, Implementation and testing during the development of software applications.
Proficiency in Database Programming using Oracle, SQL Server, HQL and MySQL creating stored procedures, Triggers, Indexes, Functions, Views, Joins etc.
Experience in Programming, designing, Testing, Deploying and Supporting equipment and systems using various programming PLC, SCADA, HMI etc.
Excellent communication and self-motivated to implement complex rules.
Quick learner and excellent team player, ability to meet deadlines.

TECHNICAL SKILLS

Streaming: Kafka, Spark Streaming, Flume

Bigdata Ecosystem: Hadoop, MapReduce, HDFS, HIVE, Impala, Sqoop, Oozie, Zookeeper, shell.

Operating Systems: Windows XP, Windows 7, Windows 8, Windows 10 and Windows Vista, Linux

Software Tools: Xilinx ISE, PSpice, Micro wind, ModelSim, MATLAB, SCADA, HMI, PLC, CRM Microsoft

Programming Languages: Scala, Java, C, C++, VHDL, Verilog, System Verilog

PROFESSIONAL EXPERIENCE

Hadoop Developer

Confidential

Responsibilities:

Extracted data from various Oracle tables to HDFS using Sqoop and transformed data using spark Rdd/Dataframe.
Loaded data from Hive external table to a dynamically partitioned (Partitioned on date) Hive managed table
Developed Hive scripts to join as well as aggregate data at various levels (positions, instruments, risk engine).
Loaded the aggregated data to Oracle tables via Sqoop.
Wrapped the Hive scripts and other housekeeping operations (shell scripts/java modules/email alerts) in an Oozie workflow.
Scheduled the entire application flow via Oozie Coordinator
Monitored the Oozie jobs via Ooozie console/Hue and health of the cluster via Ambar
Worked with Hadoop Technical Architect with respect to the selection of Hadoop tools for aggregation and transformation by developing poc's.
Coordinated with Hadoop admin teams to ensure the health of Hadoop services, user creation, hue login creation for development cluster
Connected with Horton works support team and Driven support team(Cascading)for resolving issues.
Performed Unit test during development.
Coordinated with QA team for functional testing and gave the knowledge transfer to run Oozie coordinator jobs in Hadoop cluster and monitoring via Oozie console/Hue
Coordinated with admin/build team for deploying code to UAT and production Hadoop clusters.

Hadoop Developer

Confidential

Responsibilities:

Worked in Loading and transforming large sets of structured, semi structured and unstructured data.
Worked on different file formats like Sequence files, XML files and Map files using Map Reduce Programs.
Developed Spark code using Scala and Spark-SQL/Streaming for faster processing of data
Imported and exported Data from/to Different Relational Data Sources like RDBMS, Teradata to/from HDFS using Sqoop.
Involved in creating Hive Tables, loading with data and writing Hive queries which invoke and run Map Reduce jobs in the backend.
Involved in writing APIs to Read HBase tables, cleanse data and write to another HBase table
Using Kafka created a consumer group, subscribed to the appropriate topic, and start receiving messages, validating them and writing results.
Ingested data from kafka is processed by spark by using map and reduce algorithms and pushed further to database.
Configured build scripts for multi module projects with Maven
Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
Worked on hadoop clusters using Cloudera CDH3 and CDH4