Apache Kafka – Quick Start Guide

Apache Kafka is a buzz word now and days and it lives upto its reputation. Kafka is a pub-sub messaging system in simple words, but differs a lot from traditional JMS systems.

This post a quick start guide on using Apache Kafka.

Pre-requisite

Following needs to be installed before you proceed further

  • JDK (tested with Java 1.7)
  • Kafka (choose a distribution)
  • Apache Zookeeper (used ver 3.4.6)

The rest of the tutorial is organized as

  • Starting Zookeeper
  • Starting Apache Kafka
  • Playing with topics
  • Writing a message(s) to the topic
  • Reading message(s) from the topic

Starting Zookeeper

Starting Zookeeper with default is simple. Move to the directory where Zookeeper is installed and execute the following

This assumes that you have the zoo.cfg in the conf directory.

Starting Apache Kafka

Move to the directory where Apache Kafka is installed and execute following commands

Kafka shall startup and display messages on console.

Now we have Kafka ready to be used.

Playing with Topics

Now that we have a fresh installation of Kafka, we need to create a topic before we can use it. Let's create a topic with name testTopic. Let's run the list command before and after we create the topic.

The first run of list won't return anything, as we don't have any topic yet. The next step is to create a topic and then execute the list command again.

Now it's time to generate and consume messages using the testTopic. To keep things simple, we shall use the console utilities that ship with Kafka to produce and consume the messages.

Writing messages to Topic

Here we shall use the Console producer and push few messages to the topic. Execute following on Console

This pushed 2 messages to the Topic. We kill the process by Ctrl-C

Now let's read these messages.

Reading messages from Topic

Again to read the messages we shall use the Console consumer utility. Execute following on the command prompt.

Notice the difference here. Instead of broker list, we use the Zookeeper ensemble here and ask the client to read from the beginning.

This is it for now. In subsequent posts we shall see, using Producer and Consumer example in Java.

Leave a Reply

Your email address will not be published. Required fields are marked *