We provide IT Staff Augmentation Services!

Big Data Developer Resume

NJ

SUMMARY:

  • 11+ years of IT experience along with 4 years of Apache Hadoop and Spark experience
  • Experienced Big Data Developer with good knowledge of HDFS architecture and Spark components such as Spark SQL, Spark Streaming and associated Spark API’s
  • Sound knowledge in HADOOP ecosystem components such as MapReduce, Pig, Hive, Spark, Sqoop & Cloudera
  • Experience in developing Pig Latin scripts, Hive scripts and using them in conjunction with simple user defined functions
  • Experience in writing SQOOP commands to export data from Oracle/SQL server database into HDFS for batch processing
  • Experience in analyzing high volume streaming data using Kafka
  • Experience in configuring and utilizing 5 node Kafka cluster
  • Knowledge in job/workflow scheduling and monitoring tools like Oozie
  • Possess strong knowledge in processing high volume data using RDD’s, and advanced Spark API’s such as Dataframes and Datasets
  • Hands on experience in developing business critical functions using SCALA
  • Experience in using the editor Eclipse Neon to create Scala projects with the build tool: SBT
  • Proficient knowledge of monitoring and managing Hadoop cluster using Cloudera
  • Background with traditional databases such as Oracle, SQL Server, MySQL with good understanding of ETL processes
  • Knowledge of NoSQL databases HBase
  • Experience working in Agile Methodology that includes iterations plan proposal, assisting in developing user stories and unit test case design
  • Apart from Big Data technologies, possess good experience in developing web based applications using JAVA/C#
  • Actively involved in the life cycle of project implementing the Systems, Designing, Development, Testing and Documentation

TECHNICAL SKILLS:

Big Data Eco Systems: HDFS, MapReduce, Hive, Pig, HBase, Sqoop, Oozie, Spark, Spark - SQL, Python, Spark Streaming, Kafka and Scala

No-SQL Database: HBase

RDBMS: Sql Server, Oracle, MS Access and MYSQL

Internet Programming: Java, HTML, Visual C#, ASP.NET, Web Services, Windows Services, ADO.NET

Project Management Tools: TFS, Fortify Tool, Service Now, AZDoc, HP Quality Center and Remedy

Operating System: Windows 10, Linux

PROFESSIONAL EXPERIENCE:

Confidential, NJ

Confidential,BIG DATA DEVELOPER (SPARK/SCALA)

Responsibilities:

  • Working with Business team and come up functional specification on data integration requirements
  • Set up and configure a 5 node Kafka cluster and monitor the broker activities
  • Configure Producers, consumers, set partitions and corresponding replication to handle effective fault tolerance
  • Export data from RDBMS to HDFS using Sqoop import options
  • Configured Oozie work flows to automate the data flow
  • Implement Spark best practices by invoking advanced Spark API’s such as Dataframes and Datasets
  • Monitor Spark tasks on each and every transformation and action via Spark UI and plan partition utilization accordingly for effective performance.
  • Provided design recommendations and thought leadership to stakeholders that improved review processes and resolved technical problems

Environment: HDFS, Sqoop, Oozie, Scala, Spark and Kafka

Confidential, NJ

BIG DATA DEVELOPER (SPARK/SCALA)

Responsibilities:

  • Working with Business team and come up functional specification on data integration requirements
  • Export data from RDBMS to HDFS using Sqoop import options
  • Configured Oozie work flows to automate the data flow
  • Implement Spark best practices by invoking advanced Spark API’s such as Dataframes and Datasets
  • Monitor Spark tasks on each and every transformation and action via Spark UI and plan partition utilization accordingly for effective performance.
  • Provided design recommendations and thought leadership to stakeholders that improved review processes and resolved technical problems

Environment: HDFS, Sqoop, Oozie, Scala and Spark

Confidential, WILMINGTON, DE

Analytical Risk Mitigation and Observational Reporting

Responsibilities:

  • Work closely with legacy system owners, gather requirements and propose an equivalent big data design
  • Effective utilization of spark Dataframes and memory caching techniques for processing gigabytes of data
  • Hands on experience in Scala programming involving the usage of case classes, key value pairs, application configurations and effective memory handling using sql context.
  • Follow best coding practices to achieve spark process parallelization to achieve better performance compared to legacy Mainframe/Oracle systems.
  • Monitor Spark tasks on each and every transformation and action via Spark UI and plan partition utilization accordingly for effective performance.
  • Provide design recommendations, quality analysis and thorough review followed by technical documentation

Environment: HDFS, Sqoop, Oozie, Scala and Spark

Confidential, WILMINGTON, DE

BIG DATA CONSULTANT

Responsibilities:

  • Involved in business requirement analysis. Working with Business Analysis team and users to understand data integration requirements
  • Imported data from RDBMS to HDFS using Sqoop import/export options
  • Create Hive Internal tables with appropriate partitioning and bucketing and write complex queries for data Analysis
  • Develop simple user defined functions using Eclipse Neon to access Hive scripts
  • Performing Data quality checks after data ingestion
  • Configured Oozie work flows to automate data flow
  • Developed Hadoop jobs using Java and Hive for data processing
  • Implemented best practices like partitioner, combiner and caching

Environment: HDFS, Pig, Hive, Sqoop, Oozie

Confidential, PHILADELPHIA, PA

.NET DEVELOPER

Responsibilities:

  • Involved in Knowledge transition and playback sessions with customer.
  • Involved in requirement gathering and preparation of SRS.
  • Involved in Detailed Design preparation.
  • Involved in estimation and resource plan
  • Involved in development and Unit testing, System testing.
  • Involved in defect fixing and supporting the System testing phase.

Environment: ASP.NET, C#, SQL SERVER 2005, WINDOWS XP/2000

Confidential

.NET DEVELOPER

Responsibilities:

  • Develop adhering to the application architecture following best coding practices
  • Design and Develop base claases and methods to perform authentication and authorization based on the logged in user credentials across the application modules
  • Develop base classes and methods to perform application to database connection followed by executing stored procedures
  • Design and Develop Oracle packages containing stored procedures
  • Perform Unit testing followed by logging the test results

Environment: ASP.NET, C#, SQL SERVER 2005, WINDOWS XP/2000

Confidential

.NET DEVELOPER

Responsibilities:

  • Assist in developing survey and authorizer module
  • Design and develop functions in C# to manage head count module
  • Worked as a team player enhancing smooth delivery of project adhering to best practices followed by unit testing of modules.

Environment: ASP.NET, C#, SQL SERVER 2000, WINDOWS XP/2000

Hire Now