Spring Boot | How to consume JSON messages using Apache Kafka
Last Updated :
11 Mar, 2024
Apache Kafka is a stream processing system that lets you send messages between processes, applications, and servers. In this article, we will see how to publish JSON messages on the console of a Spring boot application using Apache Kafka.
In order to learn how to create a Spring Boot project, refer to this article.
Working Steps
Step 1: Go to Spring initializer and create a starter project with the following dependency:
Spring for Apache Kafka
Step 2: Open the project in an IDE and sync the dependencies. In this article, we would be creating a student model where we would be posting the student details. Therefore, create a model class Student. Add data members and create constructor and override the toString method to see the messages in JSON format. The following is the implementation of the student class:
Student Model
// Java program to implement a
// student class
// Creating a student class
public class Student {
// Data members of the class
int id;
String firstName;
String lastName;
// Constructor of the student
// Class
public Student()
{
}
// Parameterized constructor of
// the student class
public Student(int id, String firstName,
String lastName)
{
this.id = id;
this.firstName = firstName;
this.lastName = lastName;
}
@Override
public String toString()
{
return "Student{"
+ "id = " + id
+ ", firstName = '" + firstName + "'"
+ ", lastName = '" + lastName + "'"
+ "}";
}
}
Step 3: Create a new class Config and add annotations @Configuration and @EnableKafka. Now create beans ConsumerFactory and ConcurrentKafkaListenerContainerFactory with Student class object.
Config clas
@EnableKafka
@Configuration
public class Config {
// Function to establish a connection
// between Spring application
// and Kafka server
@Bean
public ConsumerFactory<String, Student>
studentConsumer()
{
// HashMap to store the configurations
Map<String, Object> map
= new HashMap<>();
// put the host IP in the map
map.put(ConsumerConfig
.BOOTSTRAP_SERVERS_CONFIG,
"127.0.0.1:9092");
// put the group ID of consumer in the map
map.put(ConsumerConfig
.GROUP_ID_CONFIG,
"id");
map.put(ConsumerConfig
.KEY_DESERIALIZER_CLASS_CONFIG,
StringDeserializer.class);
map.put(ConsumerConfig
.VALUE_DESERIALIZER_CLASS_CONFIG,
JsonDeserializer.class);
// return message in JSON formate
return new DefaultKafkaConsumerFactory<>(
map, new StringDeserializer(),
new JsonDeserializer<>(Student.class));
}
@Bean
public ConcurrentKafkaListenerContainerFactory<String,
Student>
studentListner()
{
ConcurrentKafkaListenerContainerFactory<String,
Student>
factory
= new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(studentConsumer());
return factory;
}
}
Step 4: Create a class KafkaService with @Service annotation. This class will contain the listener method to publish the message on the console.
KafkaService Class
@Service
public class KafkaService {
// Annotation required to listen
// the message from Kafka server
@KafkaListener(topics = "JsonTopic",
groupId = "id", containerFactory
= "studentListner")
public void
publish(Student student)
{
System.out.println("New Entry: "
+ student);
}
}
Step 5: Start zookeeper and Kafka server. Now we need to create a new topic with the name JsonTopic. To do so, open a new command prompt window and change directory to the Kafka directory.
Step 6: Now create a new topic using the command given below:
bin/Kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic topic_name // for mac and linux
.\bin\windows\Kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic topic_name // for windows
Step 7: Now to run Kafka producer console, use the command below:
bin/Kafka-console-producer.sh --broker-list localhost:9092 --topic Kafka_Example // for mac and linux
.\bin\windows\Kafka-console-producer.bat --broker-list localhost:9092 --topic Kafka_Example // for windows
Step 8: Run the application and and type message on Kafka producer and press enter.
Similar Reads
How To Dockerize A Spring Boot Application With Maven ? Docker is an open-source containerization tool that is used for building, running, and managing applications in an isolated environment. It facilitates the developers to bundles its software, libraries, and configuration files. The Docker facilitates with isolating the one container from another. In
12 min read
Dynamic Dropdown From Database using Spring Boot The concept of dynamic dropdown (or dependent dropdown) is exciting and challenging to code. Dynamic dropdown means that the values in one dropdown list are dependent on the value selected in the previous dropdown list. A simple example would be three dropdown boxes displaying names of the district,
11 min read
Spring - RestTemplate Due to high traffic and quick access to services, REST APIs are getting more popular and have become the backbone of modern web development. It provides quick access to services and also provides fast data exchange between applications. REST is not a protocol or a standard, rather, it is a set of ar
7 min read
Spring Boot - Scheduling Spring Boot provides the ability to schedule tasks for execution at a given time period with the help of @Scheduled annotation. This article provides a step by step guideline on how we can schedule tasks to run in a spring boot application Implementation:It is depicted below stepwise as follows:Â St
4 min read
Spring Boot - Sending Email via SMTP Spring Boot provides the ability to send emails via SMTP using the JavaMail Library. Here we will be illustrating step-by-step guidelines to develop Restful web services that can be used to send emails with or without attachments. In order to begin with the steps, let us first create a Spring Boot p
5 min read
Different Ways to Establish Communication Between Spring Microservices If you are working with a Spring Boot project which involves multiple microservices, You might have felt the need to communicate from microservice m1 to microservice m2. Depending upon business use-cases, this communication can be of synchronous or asynchronous type. Before digging dip further. Let'
3 min read
JSON using Jackson in REST API Implementation with Spring Boot When we build REST APIs with Spring Boot, we need to exclude NULL values from the JSON responses. This is useful when we want to optimize the data being transferred, making the response more compact and easier to process for the client.In this article, we are going to learn the approach that is used
4 min read
How to encrypt passwords in a Spring Boot project using Jasypt In this article, we will learn how to encrypt data in Spring Boot application config files like application.properties or application.yml. Inside those files, we can encrypt username, password, etc. You often come across developing projects where you have to connect to databases like MongoDB, etc, a
4 min read
Containerizing Spring Boot Application Java applications, known for their robustness sometimes become difficult to deploy and manage. This is where containerization comes into the picture. Packaging your Java app into a lightweight and self-contained independent unit can provide many benefits to the developers: PortabilityScalabilityFast
3 min read
How to Send Images in Spring Boot without using Servlet and Redundant JSP Codes? Open IntelliJ IDE and in the Controller section of your project inside the Controller class you need to build the Image serve method. The Image serve method looks like below and as you have to collect the Image response use @GetMapping Annotation to collect the image. ImageHandler Controller Method
2 min read