We provide IT Staff Augmentation Services!

Hadoop Engineer Resume

Mountain View, CA

SUMMARY:

  • Excellent understanding of Hadoop core components such as HDFS, MapReduce and ecosystem such as Pig, Hive, Sqoop, Oozie, ZooKeeper and Flume.
  • Experience with Cloudera’s distribution including Hadoop (CDH) and Cloudera Manager to install, configure and monitor Hadoop clusters.
  • Good understanding of Kafka messaging system and Spark architecture.
  • Experience in deploying scalable Hadoop cluster on AWS cloud.
  • Experience with NoSQL databases such as HBase.
  • Good knowledge on cloud computing environment like Amazon Web Services (AWS) EC2, S3, Elastic Beanstalk, Elastic MapReduce (EMR), Elastic Load Balancer, Auto Scaling, Cloudfront, IAM, Redshift and CloudWatch.
  • Experience in working on large - scale big data implementations and in production environment.
  • Hands-on experience in Data Migration from Relational Database to Hadoop Platform using Sqoop.
  • Experience in using Pig scripts to do transformations, event joins, filters and some pre-aggregations before storing the data onto HDFS.
  • Experience in developing applications using Java and mobile application development technologies.
  • Experience in Linux/Unix shell scripting.
  • Experience in complete Software Development Life Cycle (SDLC) that includes Requirement Analysis, Design, Coding, Testing and Implementation using Agile (Scrum) and other development methodologies like Kanban and Waterfall.
  • Experience in issue logging and tracking using Atlassian JIRA.
  • Good communication, interpersonal skills and also a great team player with strong problem-solving skills.

TECHNICAL SKILLS:

Big data/Event Processing: Apache Hadoop, ZooKeeper, Pig, Hive, Sqoop, Oozie, Flume, Hue, Kafka, Spark

Languages: Java, Swift, Python, SQL, Pig Latin

Databases: Oracle, MySQL, HBase

Applications: Xcode, Android Studio, IntelliJ IDEA, Eclipse, NetBeans, Visual Studio,Git, Atlassian JIRA, Confluence, SQL Developer, Tableau.

Web Technologies: HTML, CSS, JavaScript, Bootstrap, JSON.

Content Management System: Clickability

Project Methodologies: Scrum (Agile), Kanban, and Waterfall

Cloud computing: Amazon EC2, S3, Elastic Beanstalk, IAM, CloudWatch

Management Tools: Maven, Jenkins, HP ALM (Quality Center)

PROFESSIONAL EXPERIENCE:

Confidential, Mountain View, CA

Hadoop Engineer

Roles & Responsibilities:

  • Worked on building Data lake, which is a fully scalable, reliable and secure cloud data platform for providing data services across the organization.
  • Worked on the legacy MapR distribution of Hadoop before migrating over to Elastic Map Reduce (EMR) distribution of Hadoop.
  • Worked on setting up S3 as the backend file system for the Hadoop cluster.
  • Working with the data engineering team on a regular basis to deploy the ETL code in production.
  • Supporting Oozie infrastructure, which is the scheduler for all our ETL jobs.
  • Supporting EMS, an internally written tool for importing data from different sources into EMR.
  • Supporting the surrounding components of the Data platform including tools like Redshift, S3, etc.
  • Working on setting up and configuring the Real-time analytics pipeline that comprises of services like Kinesis and Simple Queue service (SQS).
  • Maintaining the tableau server infrastructure in AWS.
  • Providing on-call production support during US hours (12x7)

Project specific tools and skills: EMR 5.x, Oozie, S3, IAM, Redshift, Lambda, SNS, Kinesis, SQS, Tableau 10.x, RHEL 6.5

Confidential

Hadoop Engineer

Roles & Responsibilities:

  • Identified the best solutions/Proof of Concept leveraging Big Data & Advanced Analytics that provide the customer with a robust analytics platform as a service.
  • Involved in initiating and successfully completing Proof of Concept for pre-processing, increased reliability and ease of scalability over traditional database.
  • Gathered requirements for building a distributed parallel processing analytics platform.
  • Involved actively with Research and Development (R&D) for successful Hadoop implementation.
  • Analyzed the performance of hadoop cluster at different scales thereby optimizing the cluster.
  • Supported technical team members in management and review of Hadoop log files and data backups.
  • Installed, configured and maintained the Hadoop cluster.

Project specific tools and skills: Hadoop (MapReduce), Hive, Pig, Oozie, Sqoop, Java, SQL, MySQL.

Confidential

Software Engineer

Roles & Responsibilities:

  • Designed new features to the existing client website based on user requirements using velocity scripting in Clickability tool and testing the website on different platforms with help of simulators and online tools.
  • Developed and worked on content changes, attribute changes in the templates for the hot fixes and enhancements for immediate production deployment.
  • Completed projects and work assignments on time, while aware of tight deadlines.
  • Collaborated successfully with cross-functional development teams to design and manufacture new products.

Project specific tools and skills: Velocity scripting, JIRA, FileZilla, Clickability (Content Management System), iOS simulator, browserstack, Selenium integrated through TestNG Framework

Confidential

Software Engineer

Roles & Responsibilities:

  • Gained application knowledge and business flow through analyzing client requirements, preparation of development scenarios, and providing estimations for requirements, identifying and creating test data.
  • Developed different functional features for the Editorial application using JFrame, Swing/AWT components etc.
  • Trained and provided knowledge transfer for new team members regarding the application and different integrated modules.

Project specific tools and skills: Java, Swing/AWT Components, XHTML, Eclipse, REST Client, Atlassian JIRA.

Hire Now