Kafka and Zookeeper get started
Getting started with Kafka and Zookeeper
Setup java environment and then add Apache Zookeeper and Apache Kafka. Start both and then setup local Producer and Consumer with a first stab at using topics including --from-beginning option.
sudo apt list | grep openjdk
sudo apt-cache search openjdk
sudo apt-get install -y openjdk-8-jre
Download files from:
https://kafka.apache.org/ and follow Quickstart guide (from left hand menu) and get
https://www.apache.org/dyn/closer.cgi...
https://zookeeper.apache.org/ and http://apache.mirrors.nublue.co.uk/zo... to get stable release
tar -xzf kafka_2.11-1.0.0.tgz
tar -zxf zookeeper-3.4.10.tar.gz
cd kafka_2.11-1.0.0/
./bin/zookeeper-server-start.sh config/zookeeper.properties
./bin/kafka-server-start.sh config/server.properties
On Producer
./bin-console/kafka-console-producer.sh --broker-list localhost:9092 --topic MyFirstTopic1
./bin-console/kafka-console-producer.sh --broker-list 192.168.1.235:9092 --topic MyFirstTopic1
On consumer
cd kafka_2.11-1.0.0/
./bin/kafka-console-consumer.sh --bootstrap-server 192.168.1.235:9092 --topic MyFirstTopic1
Remember we write (produce) to the Leader (which then internally writes to all replicas and if majority are successful reports back success).
We can read (consume) from any broker has they can access the data)