We provide IT Staff Augmentation Services!

Hadoop Developer Resume

SUMMARY

  • 7+ Years of experience with emphasis on Big Data Technologies, Development, Administration and Design of Java based enterprise applications.
  • Expertise in Hadoop Development/Administration.
  • Expertise in Software Development Life Cycle (Requirements Analysis, Design, Development, Testing, Deployment and Support).
  • Sounds Knowledge in databases design features including ER Diagrams, normalization, Tablets, Temporary tables, constraints, keys, data dictionaries, and data integrity
  • Experienced in setting up standards and processes for Hadoop based application design and implementation.
  • Hands on experience in installing, configuring and using ecosystem components likeHadoopMap Reduce, HDFS, Hbase, Oozie, Hive, Pig, Flume.
  • Experienced in developing Map Reduce programs using Apache Hadoop for working with Big Data.
  • Extensive Experience on importing and exporting data using stream processing platforms like Flume and Kafka.
  • Expertise in NoSQL databases including HBase.
  • Expertise in importing and exporting the data using Sqoop from HDFS to Relational Database systems/mainframe and vice - versa.
  • Expertize with the tools inHadoopEcosystem including Pig, Hive, HDFS, MapReduce, Sqoop, Spark, Kafka, Y/arn, Oozie, and Zookeeper.
  • Excellent knowledge onHadoopecosystems such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programming paradigm
  • Experience in designing and developing applications in Spark using Scala to compare the performance of Spark with Hive and SQL/Oracle
  • Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
  • Performed data analysis using Hive and Pig.
  • Loading log data into HDFS using Flume.
  • Expertise in using Sqoop, Zoo Keeper and Cloudera Manager.
  • Expertise in back-end procedure development, for RDBMS, Database Applications using SQL and PL/SQL. Hands on experience on writing Queries, Stored procedures, Functions and Triggers by using SQL.
  • Excellent communication, interpersonal, analytical skills, and strong ability to perform as part of team.
  • Exceptional ability to learn new concepts.
  • Hard working, Quick learner and enthusiastic.
  • End to end working experience in all the SDLC phases.
  • Well versed with all stages of Software Development Life Cycle (SDLC) i.e. Requirement(s) gathering, analyzing the same, Design, Implementation with usage of SQL mappings, sessions and workflows.
  • An excellent team player and self-starter with good communication skills and proven abilities to finish tasks before target deadlines.

TECHNICAL SKILLS

Programming Languages: SQL, PL/SQL,T-SQL, Databases. SQL, T-SQL, PL/SQL, C, C++, C#, CSS, HTML, Java.

Databases: NO SQL (HBase), MY SQL,MS SQL server.

IDE’s & Utilities: Eclipse and JCreator, Net Beans.

Web Dev.Technologies: HTML, XML.

Protocols: TCP/IP, HTTP and HTTPS.

Operating Systems: Windows 7,8,10, Unix, Linux, Red hat.

ETL tools: Tableau,VMplayer

Hadoop ecosystem: Hadoop and Map Reduce, Sqoop, Hive, PIG, HBASE, HDFS, Zookeeper, Oozie, and Kafka.

PROFESSIONAL EXPERIENCE

Confidential

Hadoop Developer

Responsibilities:

  • Installed and configured Hadoop MapReduce, HDFS and developed multiple MapReduce jobs in Java for data cleansing and preprocessing.
  • Evaluated business requirements and prepared detailed specifications that follow project guidelines required to develop written programs.
  • Responsible for building scalable distributed data solutions using Hadoop.
  • Analysed large amounts of data sets to determine optimal way to aggregate and report on it.
  • Handled importing of data from various data sources, performed transformations using Hive, MapReduce, and loaded data into HDFS.
  • Importing and exporting data into HDFS using Sqoop.
  • Wrote MapReduce code to make un-structured data into semi- structured data and loaded into Hive tables.
  • Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting.
  • Worked extensively in creating MapReduce jobs to power data for search and aggregation
  • Worked extensively with Sqoop for importing metadata from Oracle.
  • In depth understanding/knowledge ofHadoopArchitecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, MRv1 and MRv2 (YARN).
  • Extensively used Pig for data cleansing.
  • Created partitioned tables in Hive.
  • Managed and reviewed Hadoop log files.
  • Involved in creating Hive tables, loading with data and writing hive queries which will run internally in MapReduce way.
  • Used Hive to analyse the partitioned and bucketed data and compute various metrics for reporting.
  • Installed and configured Pig and also written PigLatin scripts.
  • Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
  • Created Hbase tables to store various data formats of data coming from different portfolios.
  • Developed MapReduce jobs to automate transfer of data from Hbase.
  • Used SVN, Tortoise SVN version control tools for code management (checkins, checkouts and synchronizing the code with repository).
  • Worked hands on with ETL process.

Environment: Hadoop, MapReduce, Hive, HBase, HDFS, Hive,Java(JDK 1.6), Cloudera, MapReduce, PL/SQL, SQL*PLUS, UNIX Shell Scripting. Java 6, Eclipse

Confidential, Oregon, OR

Hadoop Developer

Responsibilities:

  • Worked with business teams and created Hive queries for ad hoc access.
  • Continuous monitoring and managing the Hadoop cluster through Cloudera Manager.
  • Involved in review of functional and non-functional requirements
  • Responsible to manage data coming from different sources.
  • Installed and configured Hadoop ecosystem like HBase, Flume, Pig and Sqoop.
  • Loaded daily data from websites to Hadoop cluster by using Flume.
  • Involved in loading data from UNIX file system to HDFS.
  • Creating Hive tables and working on them using Hive QL.
  • Created complex Hive tables and executed complex Hive queries on Hive warehouse.
  • Wrote MapReduce code to convert unstructured data to semi structured data.
  • Used Pig to extract, transformation & load of semi structured data.
  • Installed and configured Hive and also written Hive UDFs.
  • Develop Hive queries for the analysts.
  • Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig.
  • Cluster co-ordination services through ZooKeeper.
  • Collected the logs data from web servers and integrated in to HDFS using Flume.
  • Creating Hive tables and working on them using Hive QL.
  • Worked on Hive for exposing data for further analysis and for generating transforming files from different analytical formats to text files.
  • Design and implement Map Reduce jobs to support distributed data processing.
  • Supported Map Reduce Programs those are running on the cluster.
  • Involved in HDFS maintenance and loading of structured and unstructured data.
  • Wrote Map Reduces job using Java API.
  • Designing NoSQL schemas in Hbase.
  • Wrote the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
  • Involved in Hadoop cluster task like Adding and Removing Nodes without any effect to running jobs and data.
  • Developed the Pig UDF’S to pre-process the data for analysis.
  • Involved in Hadoop cluster task like Adding and Removing Nodes without any effect to running jobs and data.

Environment: Hadoop, Map-Reduce, HDFS, Hive, Pig, HBase, Java, Cloudera Linux, XML, MySQL, MySQL Workbench, Java 6, Eclipse, Cassandra.

Confidential, Piscataway, NJ

Hadoop Developer

Responsibilities:

  • Involved in review of functional and non-functional requirements.
  • Installed and configured Pig and also written PigLatin scripts.
  • Wrote MapReduce job using Pig Latin.
  • Involved in ETL, Data Integration and Migration
  • Imported data using Sqoop to load data from Oracle to HDFS on regular basis.
  • Developing Scripts and Batch Job to schedule various Hadoop Program.
  • Written Hive queries for data analysis to meet the business requirements.
  • Creating Hive tables and working on them using Hive QL.
  • Importing and exporting data into HDFS from Oracle Database and vice versa using Sqoop.
  • Experienced indefining jobflows.
  • Got good experience with NOSQL database H-Base.
  • Involved in creating Hive tables, loading the data and writing hive queries that will run internally in a map reduce way.
  • Developed a custom File System plugin for Hadoop so it can access files on Data Platform.
  • The custom File System plugin allows Hadoop Map-Reduce programs, H-Base, Pig and Hive to work unmodified and access files directly.
  • Designed and implemented Map-reduce-based large-scale parallel relation-learning system
  • Extracted feeds form social media sites such as Facebook, Twitter using Python scripts.
  • Setup and benchmarked Hadoop/H-Base clusters for internal use

Environment: Hadoop, Map-Reduce, HDFS, Java, Hadoop distribution of Cloudera, Pig,H-base, UNIX.

Hire Now