Hadoop Developer Resume

SUMMARY

5 years of extensive Hands on experience in Big Data technologies like MapReduce (MR), Hive, Pig, HBase, Sqoop, Oozie, Flume, Spark with Scala for data extraction, processing, storage and analysis Experienced in various cloud technologies like Azure and Amazon web services. Experience in various different projects like data lake, Migration, Support and Application developments.

TECHNICAL SKILLS

PROFESSIONAL EXPERIENCE

Confidential

Hadoop Developer

Responsibilities:

Developed code for importing and exporting data from different workstreams such as data lake.
Implemented Partitions, Buckets based on State to further process using Bucket based Hive joins.
Creating Athena glue tables on existing csv data using AWS crawlers
Experience with AWS cloud services such as: EC2, EMR, RDS, Redshift
Created HBase tables and performed various operations for loading, reading the HBase table data.
Developed UDF’s in Java as and when necessary to use in Hive queries and Spark.
Handled ETL Framework in Spark for processing the data from HDFS and Hive.
Involved in Spark streaming using Scala for real - time computations to process JSON files.
Worked on Kafka messaging queue for Data Streaming in both batch and real-time applications.
Orchestrated tasks using Oozie using different actions.
Developed various scripting functionality using shell Script and Python.
Pushed application logs and data streams logs to Kibana server for monitoring and alerting purpose.
Developed Jenkins pipelines for continuous integration and deployment purpose.

Environment: Apache Hadoop 2.0, Cloudera, AWS,HDFS, MapReduce, Hive, Impala, HBase, Sqoop, Kafka, Spark, Linux, MySQL.

Confidential

Hadoop Developer

Responsibilities:

Developed and Tuned Spark Streaming application using Scala for processing data from Kafka
Imported batch data using Sqoop to load data from MySQL to HDFS on regular basis from various sources.
Extracted data from various APIs, data cleansing and processing by using Java and Scala
Used Lambda, Kinesis, Dynamo DB, cloud watch from AWS.
Knowledge on handling Hive queries using Spark SQL that integrate Spark environment.
Worked on MVP to migrate data from HDFS to Azure HD Insights.
Worked on docker and kubernetes for deploying application and make it containerize.
Developed complex Hive ETL Logic for data cleansing and transformation of data coming through relationals systems.
Implemented NiFi pipelines to export data from hdfs to different end points and cloud locations like Aws S3 .
Implemented complex data types in hive also used multiple data formats like ORC,Parquet.

Environment: Hive, Sqoop,AWS, LINUX, Cloudera CDH 5, Scala, Kafka, HBase, Avro, Spark, Zookeeper and MySQL.

Confidential

Software Engineer

Responsibilities:

Involved in designing and developing modules at both Client and Server Side.
Developed the UI using JSP, JavaScript and HTML.
Responsible for validating the data at the client side using JavaScript.
Involved in developing Server-Side Java Servlets to capture the posted form data and process it on the Server side.
Developed web components using JSP, Servlets and JDBC.
Designed the controller using Servlets.
Accessed backend database Oracle using JDBC.
Developed and wrote UNIX Shell scripts to automate various tasks.
Developed user and technical documentation.

Environment: Java, Servlets, JSP, JavaScript, JDBC, Unix Shell scripting, HTML, Eclipse, Oracle 8i, WebLogic