We provide IT Staff Augmentation Services!

Hadoop Developer Resume

3.00/5 (Submit Your Rating)

Cupertino, CA

SUMMARY

  • Passionate and proficient Big Data Engineer/Developer with around 8 years of experience in developing java applications and implementing big data solutions in Java/Scala
  • Dealing with Big data applications & NoSQL/relational database systems has been my passion. I am a technology enthusiast who keeps pace with latest technologies
  • Experience with agile development model working on complete Software Design Lifecycle including design, development, testing and implementation of moderate to advanced complex systems
  • Dedicated commitment, keen interest in learning new things and working hard to adapt myself to any challenging new requirements are my positive assets.
  • Strong in developing Map Reduce applications, MRUnit Test, analyzing and processing data using Hive and Pig, creating Map Reduce workflows.
  • Strong understanding of Hadoop ecosystem components like MapReduce, Zookeeper, Pig, Hive, Hbase, Oozie, Scoop, Flume, Cloudera Manager, Hadoop YARN, Spark
  • Good Understanding on Hadoop YARN(Yet Another Resource Negotiator) Architecture
  • Good at python scripting
  • Implemented various ETL activities using Pig, Hive and Sqoop. Worked with Sequence file and Avro formats.
  • Expertise in Object Oriented Programming using Scala, Java and J2EE related technologies
  • Excellent understanding and knowledge of NoSQL databases like HBase and Cassandra
  • Hands on experience writing Applications on Hadoop YARN architecture

TECHNICAL SKILLS

Hadoop Ecosystem: MapReduce, Spark, Scalding, Pig, Hive, Oozie, Sqoop, Flume, Hadoop YARN

No - sql Databases: Hbase, Cassandra

Indexes: Elastic Search

Languages and Technologies: Java, Scala, Python

Operating Systems: Windows & Linux

Databases: Oracle, MySQL

IDE: Eclipse, Intellij

Tools: Oracle SQL Developer, Toad, Razorsql,MsExcel

App Server: Apache Tomcat

Framework: Play Framework, Spring Framework

PROFESSIONAL EXPERIENCE

Confidential, Cupertino CA

Hadoop Developer

Environment: Scala, Map reduce, Scalding, Spark, Oozie, Cassandra, Python, Play Framework, Swagger API, Akka Schedulers

Responsibilities:

  • Worked as a part of core data team using Scala for Confidential Maps, involving loading vendor given maps data into the Confidential data model system, providing apis for data editing
  • Provided L2 support for any Hadoop map reduce bug fixes during releases/deployments
  • Extensively dealt with semi-structured json format of data
  • Experience in working with High transactional NoSQL databases like Cassandra & indexes like Elastic search
  • Dealt widely with Scala concurrent programming model involving Futures & Promises
  • Wrote complex map reduce programs for migrating old production data to a new Cassandra key space/database, while correcting the incorrect information. This involved writing custom partitioners, group comparators & sort comparators
  • Wrote MR programs to update existing production data with new features.
  • Developed MR programs to exclusively join data read from Cassandra in applications
  • Developed a light weight swagger API using Play framework with Scala, that is used by next level teams to search, edit & add new data, hiding the details of MR implementation
  • Invoked all MR applications through swagger API making Oozie REST calls
  • Used Elastic search extensively for quick search of data
  • Bulk loaded data into Cassandra key space & elastic search index using MR programs
  • Converted metadata from PostgreSQL database & csv files into json format & loaded the metadata into Cassandra & Elastic Search using MR programs
  • Converted existing few map reduce programs to scalding jobs that involved multiple intermediate reads & writes of data
  • Involved in implementing various optimization techniques that improved performance of existing map reduce applications
  • To avoid multiple reads from Cassandra in near intervals, implemented Cache in hdfs to avoid long hours of reading
  • Migrated few mapreduce applications to spark
  • Helped in writing python scripts for monitoring the health & status of various development environment instances used
  • Wrote Python scripts to search for existing ES indexes in various environments that are not in use & clean such indexes
  • Wrote Python scripts to search through Jenkins job console logs & find data for failed jobs & dispose such intermittent data immediately
  • Added smoke tests to check the expected functionality of API’s developed
  • Involved in integrating the metadata of the data loading process with multiple teams for their easy use of the programs
  • Wrote Akka Schedulers for monitoring directories hourly and process unprocessed data
  • Used Git and Gradle as code repository & build tools

Confidential, Memphis, Tennessee

Hadoop Engineer

Environment: Java, Flume, Oozie, Hive, Map Reduce, Sqoop

Responsibilities:

  • Imported data from relational databases into HDFS with Sqoop
  • Parsed cleansed and mined useful and meaningful data in HDFS using hive for further analysis
  • Performed analysis on structured and semi-structured data in HDFS and Hive
  • Fine tuning hive jobs for optimized performance using partitions and buckets
  • Experienced in analyzing data using HiveQL and custom Map Reduce programs in JAVA
  • Retrieved data from weblogs on a daily basis using Apache Flume before ingesting into HDFS for further processing using Hive
  • Extended HIVE core functionality by writing custom UDF’s
  • Experience in retrieving data from HDFS and ingesting into MYSQL and Oracle databases with Sqoop
  • Configured Flume to import the clickstream data from the consolidated servers into HDFS

Confidential, Reno, Nevada

Hadoop Consultant

Environment: Java/J2EE, Map reduce, Oozie, Pig, Hive, Sqoop, Hbase

Responsibilities:

  • Wrote scripts to monitor the health of hadoop daemon services and act accordingly
  • Experience in working with High transactional NoSQL databases like hbase
  • Imported data into Hbase using the Map reduce Java API
  • Parsed cleansed and mined useful and meaningful data in HDFS using Map-Reduce for further analysis
  • Good at understanding the Hbase Architecture for reads and writes
  • Performed analysis on structured and semi-structured data in HDFS and Hive
  • Fine tuning hive jobs for optimized performance
  • Wrote custom UDF’s and UDAF’s in Hive for record and batch processing of data,
  • Implemented Partitioning, Dynamic Partitions, Buckets in HIVE
  • Implemented the daily workflow for extraction, processing and analysis of data with Oozie.
  • Used HBase on the back-end of the application for faster read/writes of the real-time data coming from the customers .
  • Used Pig as ETL tool to do transformations, joins and some pre-aggregations before storing the data onto HDFS.
  • Synchronized the workflow for Hive, Pig, and MapReduce jobs with OOZIE.
  • Computed various metrics using Java MapReduce to calculate metrics that define user experience, revenue etc.

Confidential

Java Developer

Environment: Java, JDBC, JSP, Servlets, HTML, JUnit, Oracle10g, SQL, Eclipse IDE, XML, Spring, Hibernate

Responsibilities:

  • Followed agile methodology and involved in various phases of Software Development Life Cycle (SDLC) of the web/desktop application like Requirement gathering, Design, Analysis and Code development.
  • Developed a prototype of the application and demonstrated to business users to verify the application functionality.
  • Designed High-level Design (HLD), Functional requirements and discussed the new functionality in team meetings.
  • Used Eclipse as development environment and deployed application on Weblogic Application server
  • Developed and implemented the MVC Pattern using JSP/Servlets, spring frameworks for Web Applications.
  • Implemented server side tasks using Servlets and XML.
  • Helped developed page templates using spring framework.
  • Extensively used Core Java concepts such as Exceptions, and Collections etc in Application
  • Created and deployed web pages using HTML, JSP, JavaScript and CSS.
  • Written JUnit Test cases for performing unit testing.
  • Worked with QA team for testing and resolving defects.
  • Communicated with the third party applications for Integration.
  • Backup of project lead on updating the Handbooks.
  • Had Client interactions, Knowledge management and sharing
  • Updated the database information with the customer changed profiles
  • Developed a Login and Registration API to manage the personal information of the customer and storing them in LDAP.
  • Integrating with third party applications which are used to manage customer secured information such as Register, forgot password, forgot username, update profile etc and simultaneously mail the customer on any updates. All the redirections and mail content are manipulated based on application ID. This project implements API for Canada customers.
  • Expertise on High level design, Low level design, Coding, Unit testing, Integration testing and Load testing.
  • Provided Technical support for production environments resolving the issues, analyzing the defects, providing and implementing the solution defects. Resolved more priority defects as per the schedule.

Confidential

SAP ABAP Developer, Java Developer

Environment: SAP ABAP - SAP scripts & Adobe Forms, Java J2EE, JSP, servlets, Spring

Responsibilities:

  • Worked closely with SD functional team on solving the issues, Expertise on debugging on the issues
  • Analyzed some related technical SAP notes when it is required to implement
  • Worked on new requirements for developing new forms, reports etc, Migrated complex existing SAP Scripts to newly introduced SAP Adobe forms, Developed new SAP Adobe forms depending on the customer requirement, Provided extended support for all the developed adobe forms
  • Had working knowledge on ABAP DDIC and Reports
  • Prepared Class level and Sequence level diagrams for a Java project development
  • Used Eclipse Java IDE to develop and debug the applications.
  • Designed High-level Design (HLD), Functional requirements and discussed the new functionality in team meetings.
  • Used Eclipse as development environment and deployed application on Weblogic Application server
  • Developed and implemented the MVC Architectural Pattern using JSP/Servlets, spring frameworks for Web Applications.
  • Implemented server side tasks using Servlets and XML.
  • Involved in implementation of the Application using java.
  • Designed templates and screens in HTML and JavaScript. .
  • Involved in development of front-end applications using JSP, HTML and JavaScript.
  • Performed unit testing, system testing and integration testing
  • Developed Unit Test Cases. Used JUNIT for unit testing of the application.
  • Worked on QualityCenter (QC) for testing of the Application.
  • Involved in code and performance reviews of the code.
  • Providing input to the documentation team.

We'd love your feedback!