SUMMARY

• Around 8 years of technical expertise in complete software development life cycle (SDLC), which includes 4 years of Kafka, Microservices development using Spring boot Framework, and 4 years of Hadoop Developments on experience working with Spring boot Framework, Apache Kafka, Apache Spark with Scala, Java and Hadoop ecosystems like MapReduce (MRv1andYARN), Sqoop, Spark, Hive, Oozie and NoSQL Databases like HBase, Cassandra.
• Excellent knowledge of Microservices Architecture Design patterns.
• Implemented various Spring Boot integrations with Kafka, Rest APIs, and various storage like (MySQL, Oracle, HBase, Cassandra, and Snowflake).
• Implemented Unit and Integration tests with Test containers, Mockito & Junit.
• Excellent knowledge of Kafka Architecture.
• Hands - on expertise in implementing Spring Kafka producer, Consumers Spring Cloud streams with various message formats like Avro, JSON & XML.
• Used Apache Flume to ingest data from different sources to sinks like HDFS for unstructured data.
• Implemented custom interceptors for flume to filter data and defined channel selectors to multiplex the data into different sinks.
• Excellent knowledge of Spark Core architecture.
• Hands-on expertise in writing different RDD (Resilient Distributed Datasets) transformations and actions using Scala and Java.
• Created Data Frames and performed analysis using Spark SQL.
• Implemented Spark Streaming/Structured Streaming data from Kafka and also developed a custom MQ receiver to consume XML events from WebMD.
• Worked on File Formats like Sequence files, performed Series on AVRO, and Parquet file formats, and Managed Hadoop log files.
• Experience in writing queries in HQL (Hive Query Language), to perform data analysis.
• Created Hive External and Managed Tables.
• Implemented Partitioning and Bucketing on Hive tables for Hive Query Optimization.
• Developed Snowflake Queries for staging and piping the data streamed from Apache Kafka into the Snowflake tables.
• Experienced in writing Oozie workflows and coordinator jobs to schedule sequential Hadoop jobs.
• Integrated Flume with Kafka, using Flume both as a producer and consumer (concept of FLAFKA).
• Implemented Python DAG scripts to schedule jobs using Airflow.
• Good understanding of Databases like MySQL, Cassandra, and HBase
• Ability to write complex SQL queries to analyze structured data.
• NoSQL database, HBase. Worked on table row key design and loaded and retrieve data for real-time data processing using HBase API and performance improvements based on data access patterns.
• Knowledge of other NoSQL databases like MongoDB, and Cassandra.
• Used Sqoop to Import data from Relational Database (RDBMS) into HDFS and Hive, storing using different formats like Text, Avro, Parquet, Sequence File, ORC File, and compression codecs like Snappy and GZIP.
• Performed transformations on the imported data and exported it back to RDBMS.
• Experienced in using Docker, and Kubernetes.
• Hands-on expertise in containerizing Micro-services.
• Worked with Helm Charts to manage Kubernetes Resources objects
• Experienced in using GIT, Bit Bucket, and SVN.
• Ability to deal with build tools like Apache Maven, and SBT.
• Implemented various GraphQL Queries to perform fast and lightweight database operations on Postgres DB
• Integrated Hasura GraphQL queries with Spring boot-microservices to perform event streaming from source Kafka.
• Excellent knowledge of Object-Oriented analysis and design. Very good at analyzing the user requirements and using the design patterns.
• Designed and developed Java enterprise and web applications using Java, and JDBC API.
• Utilized the concepts of multi-threaded programming in developing applications.

TECHNICAL SKILLS

Language: Java, Scala, Linux, AWS, Azure, Frameworks - Spring bootHadoop - Apache Spark, Spark SQL, MapReduce, Hive, Kafka, Flume, Airflow, Docker, K8s Oozie, Sqoop, Jenkins, Snowflake, GraphQL

Distributions: Databricks (with AWS), MapR, Horton Works, Amazon EMR.

NoSQL DB: Cassandra, HBase, Neo4J

PROFESSIONAL EXPERIENCE

Confidential, GA

Java/Micro Services/Big Data Developer

Responsibilities:

• Good hands-on experience developing microservices using the Spring boot framework.
• Implemented various Spring Boot integrations with Kafka, Snowflake, Cassandra, HBase & SQL Databases.
• Streamed various types of Data formats like Avro, JSON, and XML from Kafka topics using Spring Kafka.
• Experienced in loading and transforming millions of events in near real-time from Kafka and writing data to Cassandra.
• Designed appropriate microservices implementation to hop data from source to target systems.
• Integrated with Machine learning model APIs with Spring Kafka to recommend programs to the Identified Risk Members.
• Implemented Spring cloud stream to subscribe to multiple topics and transformed data to ingest to Snowflake in canonical structure.
• Involved in transforming the data with the rest API calls using Spring rest controllers.
• Transferred the analyzed data across Datawarehouse from Kafka to Snowflake.
• Involved with UI team in developing automation testing using Cypress, and Karate tools.
• Containerized spring boot Microservices and deployed to Azure Kubernetes
• Developed multiple MapReduce jobs in java for data cleaning and preprocessing involved in importing and exporting data into HDFS and Hive.
• Imported data into HDFS using Sqoop.
• Experience in retrieving data from databases like MYSQL and Oracle into HDFS using Sqoop and ingesting them into HBase.
• Designed API'S to load data from google analytics and Big query.
• Design star schema in Bigquery
• Monitoring Biqquery and cloud data flow jobs.
• Using rest API with python to ingest data and some other site to bigquery.
• Developed Hive queries to analyze the data in HDFS to identify issues and behavioral patterns.
• Used PigLatin to analyze datasets and perform transformation according to business requirements.
• Experience in using NoSQL (MongoDB) data models document, key-value & wide column, or graph model.
• Involved in implementing Camunda workflow to execute business rules for each event from Kafka.
• Implemented batch jobs using Spring Schedular to trigger Adhoc basis on error events.
• Involved in creating Hasura GraphQL queries to perform various operations on Postgres DB.
• Extensive working knowledge of working various optimizing techniques and developing improved solutions.
• Work with the Internal Platform team to plan and deploy various jobs to Azure.
• Implemented SPL (Search process Languages) queries to create dashboards on Confidential to monitor live events from Kafka.
• Monitor and debug various Microservices in case of failure or performance issues.

Confidential

Java/Big Data Developer

Responsibilities:

• Build ETL batch pipelines using Spark with Scala using Spark RDDs and Datasets on MapR distribution.
• Developed Spark Scala code to integrate audit tracking information to the ELK (Elastic Search, Logstash, and Kibana) to visualize job information on Bar charts, Pie charts.
• Developed Custom IBM MQ Receiver using Spark Streaming API to Stream XML events to the NoSQL HBase Table.
• Implement Spark Structured Streaming jobs to read Avro data from Kafka topic and write data to MaprFS in Parquet format.
• To improve high efficiency writes to reach throughput from Kafka, ingested data to the Cassandra NoSQL DB.
• Configured HBase-driven configuration for any job-related properties on Spark Startup.
• Designed API'S to load data from google analytics and Big query.
• Design star schema in Bigquery
• Monitoring Biqquery and cloud data flow jobs.
• Optimize spark job performance with various native libraries using native Spark RDDs, datasets, and broadcast datasets using hash-based joins to avoid shuffling on large datasets.
• Implemented Spark integration with hive to read data from Hive external tables via Meta store.
• Implemented Kafka Connect sink to ingest data to the NoSQL Cassandra Database for specific use cases.
• Maintained audit tracking information to track end-to-end job information using the NoSQL HBase table.
• Involved in creating Hive view & External Tables pointing to Data Lake Big Data repository to perform client Analytics.
• Performed data Profiling on data from various types of database systems using SQL queries, also test for Sqoop connectivity from source databases to Big Data Systems.
• Created Oozie workflows and coordinator jobs, to schedule Spark consumption jobs, Sqoop jobs & Hive Queries.
• Implemented Spring boot microservices to look up data from Cassandra Tables and exposed API endpoints to the consumers.
• Developed Proof of concept on Graph DB Neo4J using Cypher Query Language to build simple ontology/object relationships.
• Ingested processed data into Databricks delta tables, Snowflake, and respective Salesforce PRM objects.
• Developed curl job scripts to pass parameters and run data pipeline jobs.
• Developed Jenkins automated workflows for CI/CD and get notifications to outlook.
• Worked closely with Data Science teams to gather requirements on data on the ETL, analysis phase.
• Configured and Deployed Spark streaming/batch jobs to the Kubernetes cluster within the Healthcare cloud platform.

Confidential

Java/Big Data Developer

Responsibilities:

• Worked with Hadoop Ecosystem components like HDFS, HBase, Sqoop, Hive, Spark-Scala, and Pig with HORTONWORKS Hadoop distribution.
• Utilized SparkSQL to extract and process data by parsing using Datasets or RDDs in Hive Context, with transformations and actions (map, flat Map, filter, reduce, reduceByKey).
• Effectively used GIT (version control) to collaborate with the team members.
• Worked closely with BI and Data Science teams to gather requirements on data.
• Created Oozie workflows and coordinator jobs, to schedule Spark consumption jobs, Sqoop jobs & Hive Queries.
• Worked on Eclipse with Maven for application development.
• Responsible for debugging, fixing, and testing the existing bugs related to the application
• Handled importing of data from various data sources, performed transformations using Hive, and Pig, and loaded data into HDFS.
• Performed performance tuning and troubleshooting of MapReduce jobs by analyzing and reviewing Hadoop log files.
• Creating Hive tables to import large data sets from various relational databases using Sqoop and export the analyzed data back for visualization and report generation by the BI team.
• Implemented partitioning, dynamic partitions, and buckets in HIVE.
• Participated in Daily Stand-up calls and update the progress of all the stakeholders
• Building and deployment of the project using Jenkins.

Confidential

Java Developer

Responsibilities:

• Monitor and debug Informatica components in case of failure or performance issues.
• Responsible to analyze functional specifications and preparing technical design specifications.
• Involved in all Software Development Life Cycle (SDLC) phases of the project from domain knowledge sharing, requirement analysis, system design, implementation, and deployment.
• Developed REST web services for implementing the business logic for different functionalities in the features that are developed.
• Utilized CSS, HTML, and JavaScript for the development of the front-end screens.
• Wrote Junit test cases for testing the functionality of the developed web services.
• Involved in writing the SQL queries to fetch data from the database.
• Utilized Postman for verifying the smooth workflow of the application, how the application is changing with the newly developed functionalities, and verified the output for the web services.
• User login, search & portfolio created using HTML5, CSS3, JavaScript and jQuery.
• Extensively worked on both Enterprise and Community editions of MULE ESB. Experience working with Mule API and Runtime manager and RAML.
• Designed and implemented UI layer using JSP, JavaScript, HTML, DHTML, JSON, XML, XHTML, XSL, XSLT, XSL-FO and business logic using Servlets, JSP, SWING, EJBs and J2EE framework.
• Responsible for debugging, fixing, and testing the existing bugs related to the application.
• Developed builds using continuous integration server Jenkins.
• Extensively used GIT for push and pull requests of the code.
• Actively participated in the daily scrum meetings and bi-weekly retro meetings for knowledge sharing.
• Wrote DAO classes using spring and hibernate to interact with the database for persistence.
• Used Eclipse for application development.
• Used JIRA as the task and defect tracking system.
• Followed Agile Methodologiesto manage the life cycle of the project. Provided daily updates, sprint review reports, and regular snapshots of project progress.