We provide IT Staff Augmentation Services!

Aws/etl/sr. Bigdata Developer Resume

3.00/5 (Submit Your Rating)

San Jose, CA

SUMMARY

  • Eleven (14) + years of strong IT experience in Software Analysis, Design, Development, Implementation and Testing of Object Oriented Applications, Service Oriented Architecture (SOA) and Web based Enterprise Applications using Java/J2EE on Windows platforms
  • Experience developing and deploying Cloud, BigData and Web applications for Multibrand Retail, Financial, Auto - mobile and Telecom clients
  • Expertise in Agile Scrum, Test Driven Development (TDD) and Software Development Life Cycle (SDLC) phases including Analysis, Design, Implementation, Testing and Maintenance
  • 2 years of hands on experience with Amazon AWS cloud including building AWS Lambda functions, CloudWatch, S3, Athena and EMR
  • Working knowledge of Google Cloud Platform (GCP)
  • 2 years of strong experience in API development
  • 4 years of expertise in Big Data development using Spark, HBase, Hive, MapReduce and YARN technologies
  • 3+ years of experience in building scalable data pipelines and working on Enterprise Data Lake
  • Extensive experience in BigData ETL process including integrating Hadoop into traditional ETL, accelerating the extraction, transformation and loading of massive structured and unstructured data
  • Experience working with custom Hive User Defined Function(UDF) and User Defined Aggregate Function (UDAF)
  • Experience working with Parquet file structure
  • Experience writing Hive Query Language (HQL)
  • Very good hands on experience in working with Linux Shell scripting
  • Experience with Tidal and AutoSys scheduler
  • Expertise in Unified Modeling Language (UML) in designing Use Cases, Class diagrams, Sequence diagrams using Microsoft Visio
  • Experience with JIRA in day to day work for task and bug tracking
  • Experience in using Git and SVN for version control and Maven for Project building
  • Experience using BitBucket to host Git repositories
  • Experience deploying application with Jenkins
  • Experience working with data warehouse Netezza

TECHNICAL SKILLS

Public cloud: Amazon AWS, Google Cloud Platform

Cloud Native Data Platforms: SnowFlake, EMR, Athena, S3

Big Data Eco Systems: Apache Hadoop, HDFS, Hive, AWS EMR, Apache Spark, Tez, HBase, Phoenix, Ambari, HUE

Scripting languages: Linux shell script, Python

Languages: Java, SQL, PL/SQL

IDE Tools: IntelliJ, Eclipse, DBVisualizer

BI: IBM Netezza

Application Development Methodologies: Agile, waterfall

Design Tools: Microsoft Visio

Version Control Tools: Git, Tortoise SVN

Atlassian Tools: BitBucket, Confluence, JIRA

PROFESSIONAL EXPERIENCE

Confidential, San Jose, CA

AWS/ETL/Sr. BigData Developer

Responsibilities:

  • Architecting and designing the ETL pipeline
  • Preparing a Proof of Concept (PoC) / Proof of Technology (PoT) on various use cases and giving demonstration to client
  • Develop estimate, timeline and providing status during scrum meetings
  • Developing solutions using Python 3.x - boto3, pyarrow, pandas etc.

Environment: AWS Cloud Platform, Python 3.x, GitHub, Atlassian JIRA, Confluence

Confidential, NewYork City, NY

AWS/ETL/Sr. BigData Developer

Responsibilities:

  • Engaging directly with business partners to participate in design and development of data integration/transformation solutions per functional requirements
  • Architecting and developing data pipelines to process data feeds from different sources via staging, Curated Raw (CR) layers into Enterprise Data Lake (EDL) and Customer Data Lake (CDL)
  • Designed and developed REST microservices to support various Point Of Sale (POS) and E-comm systems to create/update customer, create/update membership, customer lookup and get customer details
  • Writing AWS Lambda functions in Java to support APIs
  • Working on AWS EMR - Phoenix HBase stack to manage Operation Data Store (ODS)
  • Migrating C360 applications from legacy Hadoop platform to AWS Cloud platform
  • Migrating legacy Hive scripts into SnowFlake compatible scripts
  • Led daily touchpoint call between C360 NY team and offshore team on guiding daily tasks and goals to achieve project timelines
  • Providing support to production deployment and post-production activities
  • Version management of the source code
  • Defect logging and tracking using Atlassian JIRA
  • Cross functional communication regarding the common touchpoints

Environment: AWS Cloud Platform, AWS Lambda functions, AWS CloudWatch, AWS EMR, SnowFlake, Phoenix, HBase, REST microservices, Java 1.8, Unix Shell Scripts, Python, DBVisualizer, BitBucket, Git, Atlassian JIRA

Confidential, New York City, NY

BigData/JAVA Developer

Responsibilities:

  • Interaction with various Business Owners to understand the problem statement and get requirements
  • Working with BigData architect to build the architecture of the application and deciding best approach to in corporate BigData technologies like Spark, HBase, and Hive etc.
  • Developing Proof of Concepts (PoC) and Proof of Technology (PoT) to ensure feasibility and suitable implementation
  • Developing Spark applications for the ETL processing part of the project, working with RDDs and DataFrames wrote in Java, to aggregate, group, filter, map, join etc.
  • Writing custom Hive UDF (User Defined Function) and UDAFs (User Defined Aggregate Function) to implement the specific business logic
  • Writing Hive HQLs to insert data into partitioned and non-partitioned tables
  • Writing the ECom (Electronics Communication) messages, attachments, Bloomberg chats etc into HBase tables for the business to perform real-time analytics
  • Writing and configuring AutoSys JIL scripts to set up the box and jobs into the platform
  • Configuring Maven and Enterprise Build Jenkins for build and deployment
  • Working with DevOps to deploy the developed code into higher environments
  • Developing the Unit Test cases to test the code
  • Working with Business Analysts and QA team to test the loaded data and applications

Environment: Hadoop YARN, Spark Core, Spark SQL, Hive, Beeline, HBase, HUE, Cloudera Hadoop, Cloudera Manager, Linux, Maven, Jenkins, FileZilla, Tectia SSH Terminal, BitBucket, JIRA, Eclipse

Confidential, New York City, NY

BigData/JAVA Developer

Responsibilities:

  • Participating to JAD (Joint Application Development) sessions to analyze various Usecases and scenarios associated with business requirements
  • Building over all architecture of the application and deciding the best suitable technologies for the problem
  • Defining the workflow of the system
  • Generating statistical reports of the Hive tables including counts, average, z-score, size etc. to facilitate the enterprise and senior leadership in organization to provide health of Big Data platform, data pattern and trending information
  • Loading the generated report into Hive for other systems to use
  • Used Maven as a project building tool and Jenkins as integration build tool
  • Used Git repository as version control tool and BitBucket to host these repositories

Environment: Cloudera Hadoop, Java JDK 1.8, HBase, Hive, Beeline, Maven, Git, BitBucket, Eclipse 4.x

Confidential, New York City, NY

JAVA/BigData Developer

Responsibilities:

  • Interaction with Business Owners and Data Scientist to understand the problem statement and get requirements
  • Developed multi-phase Java MapReduce application
  • Defined the workflow for CLAY
  • Worked with Parquet file format as an input and generated csv file as an output and stored it in HDFS
  • Uploaded generated output to HBase
  • Wrote a wrapper shell script to run CLAY jar in YARN
  • Wrote Hive queries for data and metadata analysis
  • Used Maven as a project building tool

Environment: MapR Hadoop, Java JDK 1.7, HBase, Hive, Maven, Python scripts, Pandas, Unix Shell script, Apache Parquet, PyCharm, Eclipse 4.x

Confidential, New York City, NY

JAVA/BigData Developer

Responsibilities:

  • Interacted with Data Scientist to understand the problem statement
  • Used Stanford NLP POSTagging library to generate POS score
  • Wrote DBUtil to connect with MySql database to get the metadata information
  • Wrote POJO for the attribute and related information
  • Generated output as a csv file with the feed scorecard

Environment: Java JDK 1.7, Stanford NLP POSTagging, MySql, Eclipse 4.x

We'd love your feedback!