We provide IT Staff Augmentation Services!

Hadoop Developer Resume

4.00/5 (Submit Your Rating)

SUMMARY

  • Over 3 years of experience in Design, Development, Implementation, and Maintenance of Hadoop and SQL Server applications.
  • Expertise in using Hadoop (HDFS, Map Reduce) and its Ecosystems such as, Pig, Hive, Sqoop, Oozie, Zookeeper, HBase and Flume for data storage, processing and analysis.
  • Experience in writing and developing custom UDF’s for Pig and Hive, also troubleshooting issues in Map Reduce, Pig, and Hive.
  • Experience in using Spark Standalone Cluster Manager and YARN, creating a Spark Context, configuring properties, building a Spark application, RDDs, Pair RDDs, RDD operations (Transformations and Actions), RDD Partitions, RDD Persistence, Initializing Streaming Context, D - Streams, Data frames and SQL operations, check pointing, Caching, deploying and monitoring applications, and passing functions to Spark using Scala.
  • Worked on Spark-SQL, Loading and saving data using the Parquet, JSON, and relational databases.
  • Used Flume for streaming server logs into HDFS and for website log aggregation.
  • Good knowledge on monitoring and workflow scheduling tools such as Oozie and Ganglia.
  • Excellent working knowledge on configuring Hadoop Single node and Multi node Clusters.
  • Working Knowledge on NOSQL DB’s such as Mongo DB, and HBase.
  • Good knowledge on Amazon Web Service components using EC2 for computing and S3 for storage.
  • Experience in Core Java, and Web Services.
  • Good hands-on knowledge of Configuration Management and Deployment tools such as Puppet, Ansible, and IBM Urban code deploy.
  • Proficient in scripting, Git and Git work flows using Bit bucket.
  • Experience in developing Continuous Integration/ Continuous Delivery pipelines using Team city.
  • Good understanding of container Orchestration using Docker, Kubernetes and Open shift.
  • Working knowledge of Job scheduling using Autosys.
  • Experience in TSQL DDL/DML, Creating Tables, Views, Triggers, Stored Procedures, clustered, non-clustered indexes, User Defined Functions and complex SQL queries.
  • Excellent communication and interpersonal skills. Versatile team player with proven problem-solving skills.

TECHNICAL SKILLS

  • Cloudera CDH
  • Apache Hadoop (MRv1
  • MRv2)
  • Hive
  • Pig
  • Sqoop
  • HBase
  • Oozie
  • Flume
  • Phoenix
  • Zookeeper
  • Spark
  • Mongo DB
  • Cassandra
  • Unix/Linux
  • Python
  • Java
  • SQL Server 2000/2005/2008
  • SSIS
  • MS Excel
  • Toad
  • SQL Analyzer
  • ER win
  • Windows.

PROFESSIONAL EXPERIENCE

Hadoop Developer

Confidential

Responsibilities:

  • Moving Product Database data from MS SQL Server to BDPaaS (Big Data Platform as a Service) using Sqoop.
  • Creating Hive External tables as per requirement and defined with appropriate partitions, loading and transforming the structured data using Hive.
  • Creating a Rules engine and Hive External tables as per the requirements.
  • Data export, aggregation and loading Incremental Data into HDFS using Sqoop.
  • Integrating Hive with Aqua Data Studio and Advanced Query Tool using MapR-ODBC.
  • Building the Spark source code with Maven and Deploying Spark on a cluster.
  • Developed Spark applications in Eclipse and Scala IDE with Maven and SBT.
  • Loaded data from the local file system, HDFS, relational databases and Apache Cassandra.
  • Creating Hive Context, inferring schema using case classes, Loading and saving data using the Parquet, JSON, relational databases, and arbitrary sources
  • Worked on spark streaming using Kafka with Gobblin. Used Spark JSON, Log Analysis, and Xml processing.
  • Resolving Spark-Hive Integration related Issues and Kafka Integration with Spark and Hadoop.
  • Setting up New Tenant Volumes for Dev, Stage, and Prod and creating security groups.
  • Setting up security based on UNIX based POSIX groups and SQL Server Authentication based.
  • Worked on Team Foundation Server, and Rally source control systems.

Environment: MAPR, Spark, Scala, HDFS, Hive, Kafka, Gobblin, Sqoop, Oozie, Java.

Hadoop Data Analyst

Confidential

Responsibilities:

  • Creating managed and external tables using Hive with required partitions and buckets. Moved Hive tables from one database to another. Used hive for fetching JSON data and process.
  • Creating Hive UDFs for missing functionality for analytics.
  • Created Hive queries to set performance tuning parameters to increase overall performance.
  • Worked on Avro to JSON, JSON to Avro, Avro to Meta, Avro to Schema conversions, and merging all Avro files into one Avro file, and used hive to create a table from Avro file.
  • Worked on Spark RDD Transformations and Actions for Spark applications.
  • Building the Spark source code with Maven and Deploying Spark on a cluster.
  • Developing Spark applications with Maven and SBT.
  • Worked on Spark-Sql, Loading and saving data using the Parquet, JSON, and relational databases.

Environment: CDH, Spark, Scala, HDFS, YARN, Hive, Pig, Flume, Sqoop, Oracle, Java.

Big Data/ Hadoop Developer

Confidential

Responsibilities:

  • Written Map Reduce jobs in java for data cleaning and preprocessing.
  • Involved in loading data from Linux local file system to HDFS, defining job flows, managing and reviewing log files.
  • Created components such as Hive UDFs for missing functionality in Hive for analytics.
  • Created Hive queries to set performance tuning parameters to increase overall performance.
  • Migrated the data to MySQL from Hive using Sqoop.

Environment: Cloudera CDH, HDFS, MR, Hive, Sqoop, Java, MySQL.

We'd love your feedback!