Home
Apache Kafka
Friday, February 7, 2025
I. Setup environment
1 - Install Zookeeper
Download Apache Zookeeper : Download
- Upload Zookeeper to /opt
- Unzip installation file:
$ tar -xzvf apache-zookeeper-3.9.3-bin.tar.gz
- Copy and config file:
$ cd /bin/conf
$ cp zoo_sample.cfg zoo.cfg
$ vi zoo.cfg
Change dataDir=<path_storage_data>
- Start/Stop ZooKeeper:
$ cd /bin
$ ./zkServer.sh start
$ ./zkServer.sh stop
$ ./zkCli.sh // Connect to Zookeeper server
2. Make Zookeper as Services. (RockyOS 9)
Preapare zookeeper service files:
Ex: zookeeper.service
====================
#!/bin/bash
# chkconfig: 345 80 20
[Unit]
Description=Zookeeper
After=syslog.target network.target
[Service]
Type=forking
ExecStart=/opt/zookeeper/apache-zookeeper-3.9.3-bin/bin/zkServer.sh start
ExecStop=/opt/zookeeper/apache-zookeeper-3.9.3-bin/bin/zkServer.sh stop
User=hqt
Group=hqt
UMask=0007
RestartSec=10
Restart=always
[Install]
WantedBy=multi-user.target
===========================
$ cp /zookeeper.service /usr/lib/systemd/system/
$ systemctl enable zookeeper
$ systemctl daemon-reload
$ systemctl start zookeeper
$ systemctl stop zookeeper
$ systemctl status zookeeper
$chkconfig zookeeper on
if chkconfig hasn't already existed. Install as below:
$yum install chkconfig
- Download kafka: Download here
- Upload to server: /opt/kafka
- Unzip file:
$ tar -xzvf kafka_2.13-3.9.0.tgz
- Start/stop kafka:
$ bin/kafka-server-start.sh config/server.properties
$ bin/kafka-server-stop.sh
4. Make Kafka service
$ vi /usr/lib/systemd/system/kafka.service
#!/bin/bash
# chkconfig: 345 80 20
[Unit]
Description=Kafka Service
After=syslog.target network.target
[Unit]
Requires=zookeeper.service
After=zookeeper.service
[Service]
Type=simple
ExecStart=/bin/sh -c '/opt/kafka/kafka_2.13-3.9.0/bin/kafka-server-start.sh /opt/kafka/kafka_2.13-3.9.0/config/server.properties > /opt/kafka/kafka.log 2>&1'
ExecStop=/opt/kafka/kafka_2.13-3.9.0/bin/kafka-server-stop.sh
Restart=on-abnormal
User=hqt
Group=hqt
UMask=0007
RestartSec=10
Restart=always
[Install]
WantedBy=multi-user.target
$ systemctl enable kafka
$ systemctl stat kafka
$ chkconfig kafka on
II. Kafka knowledge
// Create topic
$ bin/kafka-topics.sh --create --topic quickstart-events --bootstrap-server localhost:9092
//Write something to topic
$ ./kafka-console-producer.sh --topic quickstart-events --bootstrap-server localhost:9092
//Read content of topic
$ ./kafka-console-consumer.sh --topic quickstart-events --from-beginning --bootstrap-server localhost:9092
Write "hello" to the topic:
Read content from the topic:
is updating...
Comments[ 0 ]
Post a Comment