In the last post What is Apache Kafka, we discuss Kafka producer, Topics and Consumers.
In this post, I am going to provide JAVA code for writing Kafka Producer and explain how it works.
In this code, we are going to send data from CSV row by row to Kafka Topics for further consumption by Kafka Consumer. In this way, we can generate the continuous streams of Data.
1. First, you need to create a Kafka Topic. You can either do it from the console or do programmatically.
Please go through the basic Quickstart from Apache Kafka Website and create Topics, send basic message from producer and consume it at consumer.
Once, you have created Topic, its time to write Kafka Producer, which will send data to Topic.
For creating Procuder we need to configure it some parameters.
Lets, look at the configurations required for creating producer
1. List of Kafka Brokers
2. Serializer used for sending data to Kafka
3. Acknowledge from Kafka that messages are properly received.
In JAVA, we will use the properties for building the configuration.
We have created a KakfaProducer class to send data from the producer.
In this post, I am going to provide JAVA code for writing Kafka Producer and explain how it works.
In this code, we are going to send data from CSV row by row to Kafka Topics for further consumption by Kafka Consumer. In this way, we can generate the continuous streams of Data.
1. First, you need to create a Kafka Topic. You can either do it from the console or do programmatically.
Please go through the basic Quickstart from Apache Kafka Website and create Topics, send basic message from producer and consume it at consumer.
Once, you have created Topic, its time to write Kafka Producer, which will send data to Topic.
For creating Procuder we need to configure it some parameters.
Lets, look at the configurations required for creating producer
1. List of Kafka Brokers
2. Serializer used for sending data to Kafka
3. Acknowledge from Kafka that messages are properly received.
In JAVA, we will use the properties for building the configuration.
We have created a KakfaProducer class to send data from the producer.
After that, we will be going to initialize the configuration of a file to be read (CSV) using BufferedReader class to get the header to initialize the CSV parser.
Once, we did that its time to send the CSV data to Kafka Topic on Kafka Broker. we are going to parse the CSV and send row by row iteratively from Kafka Producer to the Kafka Topic.
And Finally, the Main class to send CSV file and initialize config.
In the next post, I will be going to give JAVA code for consuming above CSV data from Topic using Consumer.
Also, check out some more posts related to Big Data, Java and Python.
Comments
Post a Comment