How to start an event driven microservice backend application using Spring Kafka

-

Have you ever started a project from the start and it became a very large Project? 

As your application grows, the complexities to your code also grows. Even the once perfect, orderly project can turn into chaos because you constantly put more and more features, services and assets into the project, making maintenance a nightmare. 

One of the solutions to prevent a growing application from becoming a mess is by separating your application into smaller microservices. Microservice is a contrast to monolithic architecture, rather than putting your feature into one giant codebase, in microservice you divide your project into smaller projects(microservice), and then each microservice has only one responsibility with a specific task or functionality.

Here’s some feature what most application has that can be divided into microservices:

1. User authentication and user Management

Make your authentication and authorization of your APIs into different services, usually this type of service called Single sign on(SSO),  using some techniques like producing JWT tokens that can be used for authentication located on other services. Other than signing in, you can also put your user registration, user details and signing out combined in this service.

2. Website and app information

You can separate your company, web or app  information management into specific services, so that it makes it easier to maintain the code related to this feature. You can put features like managing pages, links, blogs in this type of microservice.

3. Product management Microservice

Whatever is related to the product must be handled by this microservice. This service provides APIs for listing, sorting, and product searching. Also handle the creation and updating of products.

4. Order Management

Processing and tracking orders can be isolated into a microservice. You can handle product checkout, order listing, order status updates, and related operations in this service.

5. Payment Gateway

Integration with external payment providers, managing transactions, and handling payment-related operations can be dedicated to this microservice.

6. Analytics and Reporting

Collecting and analyzing application usage data, generating reports, and providing insights can be performed by a separate microservice. This service aggregates and processes data, enabling businesses to gain valuable insights.

7. File Management

Handling image or file uploads, storage, retrieval, and serving can be abstracted into a dedicated microservice. You can utilize some cloud storage solutions or have your own storage infrastructure.

Microservice Spring and Kafka

When it comes to building scalable, maintainable backend systems, the Spring framework is one of the most popular choices not only in the Java ecosystem, but considered to be the perfect solution out of many frameworks in many programming environments.

In this article, we will develop a starting point on the journey of event-driven microservices using Kafka.

With Spring, you have the flexibility to choose the modules that best suit your needs. It provides seamless integration with various messaging systems, and one standout option is Kafka. Kafka is the right distributed streaming platform because Kafka is very excels at handling high-throughput, fault-tolerant, and scalable event streams.

This is a guide process step by step to start your Microservice in spring:

1. Setting Up Kafka Broker

First things first, we need to set up our development environment. Course you must have a kafka broker for your message streaming. Kafka broker is  a single instance of the Kafka server. It is just a running Kafka process that handles the storage of your message. Your kafka message is distributed into topics and then your message is consumed by a kafka group.

Best practice to set up a kafka for development is just by using Docker. Use this simple docker compose:

To run or start your Kafka broker for development, use the following docker command on the same folder where you put the docker-compose.yml:

docker-compose up -d

Open your docker, if you are seeing any Kafka broker in your docker container, it means that you have successfully run Kafka. Now that your kafka is running on your machine, let’s continue to configure it into your Spring project. 

So let's dive into the exciting part of coding our backend application development, with event driven microservice architecture.

2. Implementing Microservices with Spring Boot

With a kafka server running and a solid foundation about micro service provided above, it's time to get our hands dirty by starting to write our example microservice. We will be implementing the Product Microservice based on the chart above, a simple project consuming  product ID and producing and producing the ID to another topic.

To use Kafka in Spring-boot project, add Kafka in your maven dependency list.

And then set your kafka configuration inside application.yml, here’s an example of Kafka configuration in Spring boot, in the local machine, you can just use localhost. If you run Kafka on the server, change the bootstrap server with your server IP address.

Now comes to the Java part, integrating Spring Kafka into our microservices by writing  consumers and producers. In this example you will learn how to consume events within your microservices and how to publish events to Kafka topics, enabling seamless communication and event-driven behavior.

3. Testing

A robust backend application requires rigorous testing. In this tutorial I will provide you with a simple testing of our application. It will test the process of testing your event-driven microservices, ensuring that they perform as expected. Here’s how a simple  test can be written.

The above code is to test the processing of the kafka event method inside the MyKafkaService. When our application is running, processMessage will be triggered each time there’s an event coming into the productTopic.

In the next chapter we'll also discuss deployment strategies and best practices for scaling your system to handle increasing loads, so stay tuned.

This is just your first journey of event-driven microservices using Spring Kafka. Microservice means breaking down the application into smaller, independent services, you now maintain a project that has good levels of scalability and maintainability. With this new knowledge, you're well-equipped to the next level, I hope you can do more implementation of building scalable backend systems. Go fort and start building more amazing applications that you think will have a positive impact, but always remember that you must always love the process of building it.

Popular posts from this blog

ERROR 1348 Column Password Is Not Updatable When Updating MySQL Root Password

-
Image
MySQL and MariaDB password As already mentioned in the title of this blog, this error message sometimes occurred when we are trying to update our root password database, either MySQL or MariaDB, The error says: ERROR 1348 (HY000): Column 'Password' is not updatable This is because if we use this SQL command to update our database root password, and it turn out to be Restricted by MySQL to use update on mysql database. UPDATE mysql.user SET Password=PASSWORD('1234') WHERE User='root'; The solutions for MySQL Error1348 So you can not update a user password using Update commands, to update the password, use ALTER commands instead. like the following. ALTER USER 'root'@'localhost' IDENTIFIED BY '1234'; And then you need to do the flush privileges to commit previous queries (that alter command above) into the system, simply do like this. flush privileges; So now you have your root with password 1234. Although it is recommended to...
Read more

How To Create Spring Boot Project Using Netbeans

-
Image
Spring Boot is a handy tool that simplifies the development of Spring Framework applications by reducing the need for cumbersome configuration. In this tutorial, I'll guide you through creating a Spring Boot application using the Netbeans IDE. While you can certainly create a Spring Boot application in Netbeans manually, it's much easier with a plugin designed for Spring development. Here's a step-by-step guide to get you started: 1. Install the Spring Boot Plugin: Open Netbeans and navigate to the Tools menu. Click on "Plugins." Once the Plugins window opens, select the "Available Plugins" tab. Look for the "Nb SpringBoot" plugin, then install and activate it. 2. Create a New Spring Boot Project: After restarting Netbeans IDE, you can create a new project with Spring Boot. Go to File > New Project. Under Categories, select Maven, then choose "Spring Boot Basic." 3. Add Dependencies: With your new project created, open the po...
Read more

Spring Kafka - How to use ReplyingKafkaTemplate send and reply synchronously

-
Image
Microservice Spring with Apache Kafka Today, I want to share how to use Kafka. It's quite interesting once you get the hang of it, and I hope my explanation will make it a bit easier for you. So, Kafka is primarily known as an asynchronous messaging queuing system. This means it allows different parts of your application to communicate with each other without waiting for immediate responses. However, there are times when you might need a more immediate reply from another service in your project. Luckily, Kafka can handle this too. Here’s how it works: The client side sends a message to the server side via Kafka. This message includes a special identifier in the Kafka header request, known as KafkaHeaders.CORRELATION_ID, along with a topic where the server should respond. The server, after processing the request, sends a message back to the same topic with the same correlation ID. This way, the client knows that the message is a response to its initial request and can act accordingl...
Read more