We provide IT Staff Augmentation Services!

Spark Developer Resume

5.00/5 (Submit Your Rating)

Cherry Hill, NJ

SUMMARY

  • 6 years senior developer with 3 years as a Hadoop/Spark Developer in Big Data domain and 3 years as a Java developer.
  • Experienced in analyzing the different types of data flow from source to Hadoop Clusters.
  • Good at with Big Data Hadoop core and Eco - System components (HDFS, MapRedue, Hive, Yarn, Sqoop, Oozie, Hbase and Pig).
  • Hands on Cloudera and Hortonworks Distribution System of Hadoop.
  • Experienced in manipulating the streaming data to clusters through Kafka and Spark-Streaming.
  • Extensive experience in partitioning Data according to the business requirements using Hive.
  • Working with data importing and exporting in Sqoop.
  • Good knowledge in data extraction, transformation and loading between Hive, HDFS and RDBMS.
  • Proficient to work with data transformation from HDFS, HIVE, HBase, and RDBMS.
  • Experienced in creating UDF's, UDAF's for Hive.
  • Acumen in choosing an efficient ecosystem in Hadoop and providing the best solutions to Big Data problems.
  • Extensive Knowledge in Spark, Spark-Streaming, SparkSQL and Scala.
  • Proficient in working with NoSQL database like Cassandra and HBase.
  • Strong expertise in algorithm and data structure.
  • Strong ability to write complex sql query.
  • Prolific in generating the splendid and informative dashboards for Business Intelligence teams.
  • Experienced in using Front-end technologies like JavaScript, HTML, CSS, HTML5 and CSS3.
  • Experienced in Test such as JUnit
  • Proficient in relational databases like Oracle, MySQL and SQL Server.
  • Used to work in Agile environment.
  • Comfortable to program in Linux, UNIX environments. And Expertise in Unix-Shell.
  • Extensive experience with Front-end and Back-end IDE's like Eclipse, Visual Studio.
  • Proficient communication skills with an ability to lead a team & keep them motivated.
  • Good knowledge to work on AWS, including S3, EMR, EC2.
  • Highly motivated and versatile team player with the ability to work independently & adapt quickly to new emerging technologies.

TECHNICAL SKILLS

Hadoop Ecosystem Framework: MapReduce, Hive, Pig, Hadoop, Spark1.6 and 2.0 HBase, SqoopHDFS, Kafka

Databases Methodologies: Oracle, MySQL, SQL Server, Agile Scrum Hbase, Cassandra, MongoDB

Languages: Java, Scala, C++, SQL, HiveQL, HTML 5, JSON

Web Technologies: Pig Latin, JavaScript, Shell-Scripting, XML, CSS 3, Angular.JS

Systems Other: Windows, Linux, UNIX Eclipse, Maven, JUnit, Git

PROFESSIONAL EXPERIENCE

Confidential, Cherry Hill, NJ

Spark Developer

Responsibilities:

  • Involved in designing hadoop ecosystem pipeline.
  • Designed real-time data importing method as producer to pull the data from the sources to Kafka clusters.
  • Pulled the consumption records form Kafka to Spark.
  • Realized collecting incoming data in real-time and processing them within few seconds at the same time using Spark-Streaming and SparkSQL.
  • Implemented ML model to filter the records and get the fraud events.
  • Aggregated and stored the data result into HDFS and Cassandra .
  • Accomplished different goals of the management team by implement algorithms and business logics of data process in Spark related technologies.
  • Involved in using RDBMS, sqoop and Hive to do ETL jobs and some data analytics on historical records stored in RDBMS.
  • Designed some UDFs with Java to improve query performance in Hive.
  • Strong problem solving experience on test, identify, address, debug and then resolve technical issues that affect the integrity of the application.
  • Expertized in improving the performance of application by analytical skills.
  • Worked in the agile Environments, effectively communicated at all levels of an organization in Management and Technical roles.

Environment: Hadoop, Spark, Apache Kafka, RDBMS, Hive, HDFS, Cassandra, Scala, Parquet, Linux.

Confidential, NYC, NY

Hadoop/Spark Developer

Responsibilities:

  • Developed a data pipeline using Kafka, Spark, RDBMS, HDFS, Scala, MapReduce, Hive and Sqoop to ingest, transform and analyzing customer behavioral data.
  • Imported the data from RDBMS into HDFS using Sqoop.
  • Designed dynamically storage logic of data and improved performance with HDFS and Hive.
  • Designed and Developed MapReduce in Java to process large volume of data.
  • Used Kafka to collect streaming data from customers' consumption records. Configured Kafka to read and write messages from external data sources.
  • Realized process of real time streaming data using Spark Streaming and Kafka, storage data into HDFS.
  • Exported the analyzed data from HDFS to Hive using Sqoop to further visualize and generate reports for the BI team.
  • Utilized Spark Core, Spark Streaming and Spark SQL API for faster processing of data instead of using MapReduce in Java.
  • Save the final result back into HDFS and then imported to Hive.

Environment: Kafka, Spark, Scala, Cloudera, Hadoop, RDBMS, Hive, Sqoop, Java, HDFS, Linux, HBase

Confidential, NY

BigData Developer

Responsibilities:

  • Implemented Pig scripts to clean and accomplish data preprocessing.
  • Involved in importing and exporting data using Sqoop from HDFS to RDBMS and vice-versa.
  • In order to improve performance, utilize Hive scripts to reorganize data.
  • Migrated data from the sources such as HDFS and Hive into Map Reduce.
  • Developed Map Reduce programs in Java to process data.
  • Wrote various Queries on data according to business requirement.
  • Effectively communicated at all levels of an organization’s Management and Technicians.
  • Working closely with BI team.

Environment: Oracle, Linux, Hadoop, Hive, Pig, Sqoop, HBase, Java, JSON, Hortonworks

Confidential

Web Developer

Responsibilities:

  • Developed database schema and SQL queries for querying, inserting and managing database
  • Developed pages using HTML5, CSS3 and JavaScript as part of the presentation layer
  • Hibernate framework is used in persistence layer for mapping an object-oriented domain model to database.
  • Used Maven scripts to fetch, build, and deploy application to development environment
  • Used JUnit for functional and unit testing code.

Environment: JavaScript, MySQL, JUnit, CSS 3, HTML 5, RDBMS, Hibernate

Confidential

Java Developer

Responsibilities:

  • Used Apache Tomcat as the web server of the house reservation records.
  • Implemented JDBC in Java to connect Apache Tomcat and Oracle.
  • Implemented JDBC to connect User interface and Apache Tomcat.
  • Created and developed database with Oracle, participated in manipulation of data transaction and improve the performance of data
  • Developed of application using Spring.MVC to handle the data communication from User interface to Server then to Database and vice-versa.
  • Involved in exchange of data between the Client, Server and Database in Hibernate.

Environment: Eclipse, JDBC, Spring.MVC, JavaScript, JUnit, Java6, Servlets, CSS3, HTML 5, Oracle.

We'd love your feedback!