文章目录
环境
- RHEL 9.3
- Docker Community 24.0.7
- minikube v1.32.0
ZooKeeper简介
Apache ZooKeeper是一个分布式的开源协调服务，用于分布式系统。ZooKeeper允许你读、写数据以及发现数据更新。数据按层次结构组织在文件系统中，并复制到ensemble（ZooKeeper服务器集合）中所有的ZooKeeper服务器。对数据的所有操作都是原子的和顺序一致的。ZooKeeper通过Zab共识协议在ensemble的所有服务器之间复制状态机，来确保这个特性。
Ensemble使用Zab协议选举一个leader，在选举出leader前不能写入数据。选举出leader后，ensemble使用Zab来确保所有写操作被复制到一个quorum，然后这些写操作才会被确认并对客户端可见。如果没有遵照加权quorums，quorum是包含当前leader的ensemble的主要组件。 例如，如果ensemble有3个服务器，一个包含leader和另一个服务器的组件构成一个quorum。如果ensemble不能达成一个quorum，数据将不能被写入。
ZooKeeper在内存中保存它们的整个状态机，并把每个改变都写入一个持久的WAL（Write Ahead Log）。当一个服务器宕机时，它能够通过回放WAL恢复之前的状态。为了防止WAL无限制的增长，ZooKeeper服务器会定期的将内存状态快照保存到存储介质。这些快照能够直接加载到内存，快照之前的所有WAL条目都可以被丢弃。
准备
清理环境：
minikube delete --all
重启电脑。
启动minikube：
minikube start
确认环境干净：
$ kubectl get all
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 11m
$ kubectl get pvc
No resources found in default namespace.
$ kubectl get pv
No resources found
创建文件
zookeeper.yaml
如下：
apiVersion: v1
kind: Service
metadata:name: zk-hs
labels:app: zk
spec:ports:-port:2888name: server
-port:3888name: leader-election
clusterIP: None
selector:app: zk
---apiVersion: v1
kind: Service
metadata:name: zk-cs
labels:app: zk
spec:ports:-port:2181name: client
selector:app: zk
---apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:name: zk-pdb
spec:selector:matchLabels:app: zk
maxUnavailable:1---apiVersion: apps/v1
kind: StatefulSet
metadata:name: zk
spec:selector:matchLabels:app: zk
serviceName: zk-hs
replicas:3updateStrategy:type: RollingUpdate
podManagementPolicy: OrderedReady
template:metadata:labels:app: zk
spec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:-labelSelector:matchExpressions:-key:"app"operator: In
values:- zk
topologyKey:"kubernetes.io/hostname"containers:-name: kubernetes-zookeeper
imagePullPolicy: Always
# image: "registry.k8s.io/kubernetes-zookeeper:1.0-3.4.10"image:"docker.io/kaiding1/kubernetes-zookeeper:1.0-3.4.10"resources:requests:memory:"1Gi"cpu:"0.5"ports:-containerPort:2181name: client
-containerPort:2888name: server
-containerPort:3888name: leader-election
command:- sh
--c
- "start-zookeeper \
--servers=3 \
--data_dir=/var/lib/zookeeper/data \
--data_log_dir=/var/lib/zookeeper/data/log \
--conf_dir=/opt/zookeeper/conf \
--client_port=2181 \
--election_port=3888 \
--server_port=2888 \
--tick_time=2000 \
--init_limit=10 \
--sync_limit=5 \
--heap=512M \
--max_client_cnxns=60 \
--snap_retain_count=3 \
--purge_interval=12 \
--max_session_timeout=40000 \
--min_session_timeout=4000 \
--log_level=INFO"
readinessProbe:exec:command:- sh
--c
-"zookeeper-ready 2181"initialDelaySeconds:10timeoutSeconds:5livenessProbe:exec:command:- sh
--c
-"zookeeper-ready 2181"initialDelaySeconds:10timeoutSeconds:5volumeMounts:-name: datadir
mountPath: /var/lib/zookeeper
securityContext:runAsUser:1000fsGroup:1000volumeClaimTemplates:-metadata:name: datadir
spec:accessModes:["ReadWriteOnce"]resources:requests:storage: 10Gi
注：因为访问不了
registry.k8s.io
，所以事先把image pull下来，并push到了可访问的位置。
部署
尝试1
$ kubectl apply -f zookeeper.yaml
service/zk-hs created
service/zk-cs created
poddisruptionbudget.policy/zk-pdb created
statefulset.apps/zk created
检查pod：
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
zk-0 1/1 Running 0 57s
zk-1 0/1 Pending 0 35s
进入
zk-0
pod：
kubectl exec -it zk-0 -- bash
查看当前身份以及
/var/lib/zookeeper
目录：
zookeeper@zk-0:/$ whoami
zookeeper
zookeeper@zk-0:/$ ls-l /var/lib
total 0
drwxr-xr-x. 1 root root 42 Jun 13 2017 apt
......
drwxrwxrwx. 3 root root 18 Feb 6 07:13 zookeeper
zookeeper@zk-0:/$ ls-l /var/lib/zookeeper/
total 0
drwxr-xr-x. 4 zookeeper zookeeper 46 Feb 6 07:13 data
可见，
/var/lib/zookeeper
目录是
777
，所以
zookeeper
能在此创建目录。
退出容器，再次查看pod：
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
zk-0 1/1 Running 0 7m37s
zk-1 0/1 Pending 0 7m15s
可见，
zk-1
始终处于
Pending
状态。
$ kubectl describe pod zk-1
......
Events:
Type Reason Age From Message
-------------------------
Warning FailedScheduling 8m4s default-scheduler 0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
Warning FailedScheduling 2m39s (x2 over 8m3s) default-scheduler 0/1 nodes are available: 1 node(s) didn't match pod anti-affinity rules. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod..
可见，在affinity规则下，没有符合条件的node。
尝试2
清理环境：
$ kubectl delete -f zookeeper.yaml
service "zk-hs" deleted
service "zk-cs" deleted
poddisruptionbudget.policy "zk-pdb" deleted
statefulset.apps "zk" deleted
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
datadir-zk-0 Bound pvc-f4a42476-b3e7-48ca-ad8d-b8f6ec278f1e 10Gi RWO standard 15m
datadir-zk-1 Bound pvc-924378b6-5810-4ebc-86a7-90037221381f 10Gi RWO standard 14m
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-924378b6-5810-4ebc-86a7-90037221381f 10Gi RWO Delete Bound default/datadir-zk-1 standard 14m
pvc-f4a42476-b3e7-48ca-ad8d-b8f6ec278f1e 10Gi RWO Delete Bound default/datadir-zk-0 standard 15m
$ kubectl delete pvc datadir-zk-0 datadir-zk-1
persistentvolumeclaim "datadir-zk-0" deleted
persistentvolumeclaim "datadir-zk-1" deleted
修改
zookeeper.yaml
，把
requiredDuringSchedulingIgnoredDuringExecution
改为
preferredDuringSchedulingIgnoredDuringExecution
。
......spec:affinity:podAntiAffinity:preferredDuringSchedulingIgnoredDuringExecution:-weight:100podAffinityTerm:labelSelector:matchExpressions:-key:"app"operator: In
values:- zookeeper
topologyKey:"kubernetes.io/hostname"......
$ kubectl apply -f zookeeper2.yaml
service/zk-hs created
service/zk-cs created
poddisruptionbudget.policy/zk-pdb created
statefulset.apps/zk created
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
zk-0 1/1 Running 0 2m1s
zk-1 1/1 Running 0 99s
zk-2 1/1 Running 0 77s
$ kubectl get sts
NAME READY AGE
zk 3/3 6m31s
注：如果又遇到pod
Pending
问题，有以下错误：
Events:
Type Reason Age From Message
-------------------------
Warning FailedScheduling <unknown> default-scheduler 0/1 nodes are available: 1 Insufficient memory.
Warning FailedScheduling <unknown> default-scheduler 0/1 nodes are available: 1 Insufficient memory.
则需要先
minikube delete
，然后在启动minikube时，增大内存，比如：
minikube start--memory='7500mb'
查看存储：
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
datadir-zk-0 Bound pvc-7b501343-085f-48d8-b88a-ade5686b26c9 10Gi RWO standard 7m18s
datadir-zk-1 Bound pvc-4af3ab29-a3c8-4b26-addf-40656e998021 10Gi RWO standard 6m56s
datadir-zk-2 Bound pvc-60869d4a-389d-4f86-8446-4fda9858fe40 10Gi RWO standard 6m34s
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-4af3ab29-a3c8-4b26-addf-40656e998021 10Gi RWO Delete Bound default/datadir-zk-1 standard 6m58s
pvc-60869d4a-389d-4f86-8446-4fda9858fe40 10Gi RWO Delete Bound default/datadir-zk-2 standard 6m36s
pvc-7b501343-085f-48d8-b88a-ade5686b26c9 10Gi RWO Delete Bound default/datadir-zk-0 standard 7m20s
验证
进入容器：
kubectl exec -it zk-0 -- bash
检查连通性：
zookeeper@zk-0:/$ echo"Are you ok? $(echo ruok | nc 127.0.0.1 2181)"
Are you ok? imok
检查模式：
zookeeper@zk-0:/$ echo srvr | nc localhost 2181
Zookeeper version: 3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
Latency min/avg/max: 0/0/0
Received: 123
Sent: 122
Connections: 1
Outstanding: 0
Zxid: 0x0
Mode: follower
Node count: 4
可见，pod
zk-0
是一个follower。
退出容器，用同样的方法检查另外两个pod：
zookeeper@zk-1:/$ echo srvr | nc localhost 2181 | grep -i mode
Mode: leader
zookeeper@zk-2:/$ echo srvr | nc localhost 2181 | grep -i mode
Mode: follower
在leader pod里：
zookeeper@zk-1:/$ echo dump | nc localhost 2181
SessionTracker dump:
Session Sets (0):
ephemeral nodes dump:
Sessions with Ephemerals (0):
zookeeper@zk-1:/$ echo stat | nc localhost 2181
Zookeeper version: 3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
Clients:
/127.0.0.1:60780[0](queued=0,recved=1,sent=0)
Latency min/avg/max: 0/0/0
Received: 221
Sent: 220
Connections: 1
Outstanding: 0
Zxid: 0x100000000
Mode: leader
Node count: 4
zookeeper@zk-1:/$ echo envi | nc localhost 2181
Environment:
zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
host.name=zk-1.zk-hs.default.svc.cluster.local
java.version=1.8.0_131
java.vendor=Oracle Corporation
java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
java.io.tmpdir=/tmp
java.compiler=<NA>
os.name=Linux
os.arch=amd64
os.version=5.14.0-362.18.1.el9_3.x86_64
user.name=zookeeper
user.home=/home/zookeeper
user.dir=/
zookeeper@zk-1:/$ echo conf | nc localhost 2181
clientPort=2181
dataDir=/var/lib/zookeeper/data/version-2
dataLogDir=/var/lib/zookeeper/data/log/version-2
tickTime=2000
maxClientCnxns=60
minSessionTimeout=4000
maxSessionTimeout=40000
serverId=2
initLimit=10
syncLimit=5
electionAlg=3
electionPort=3888
quorumPort=2888
peerType=0
zookeeper@zk-1:/$ echo mntr | nc localhost 2181
zk_version 3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
zk_avg_latency 0
zk_max_latency 0
zk_min_latency 0
zk_packets_received 242
zk_packets_sent 241
zk_num_alive_connections 1
zk_outstanding_requests 0
zk_server_state leader
zk_znode_count 4
zk_watch_count 0
zk_ephemerals_count 0
zk_approximate_data_size 27
zk_open_file_descriptor_count 41
zk_max_file_descriptor_count 1048576
zk_followers 2
zk_synced_followers 2
zk_pending_syncs 0
事实上，在spec中，使用了
zookeeper-ready 2181
来作为
readinessProbe
和
livenessProbe
：
......readinessProbe:exec:command:- sh
--c
-"zookeeper-ready 2181"initialDelaySeconds:10timeoutSeconds:5livenessProbe:exec:command:- sh
--c
-"zookeeper-ready 2181"initialDelaySeconds:10timeoutSeconds:5......
查看
zookeeper-ready
：
zookeeper@zk-1:/$ which zookeeper-ready
/usr/bin/zookeeper-ready
zookeeper@zk-1:/$ cat/usr/bin/zookeeper-ready
#!/usr/bin/env bash# Copyright 2017 The Kubernetes Authors.## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at## http://www.apache.org/licenses/LICENSE-2.0## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License.# zkOk.sh uses the ruok ZooKeeper four letter work to determine if the instance# is health. The $? variable will be set to 0 if server responds that it is # healthy, or 1 if the server fails to respond.
OK=$(echo ruok | nc 127.0.0.1 $1)if["$OK" == "imok"]; then
exit 0
elseexit 1
fi
可见，它就是用上面的方法来检测连通性的。
深入了解ZooKeeper
leader和follower
每个ZooKeeper ensemble里的服务器节点都有一个唯一的ID与其网络地址关联。每个node都知道该ID。
hostname：
$ for i in 0 1 2;do kubectl exec zk-$i-- hostname; done
zk-0
zk-1
zk-2
ZooKeeper node在数据目录的
myid
文件中存储其服务器ID（本例中数据目录是
/var/lib/zookeeper
）。
$ for i in 0 1 2;doecho"myid zk-$i"; kubectl exec zk-$i--cat/var/lib/zookeeper/data/myid; done
myid zk-0
1
myid zk-1
2
myid zk-2
3
注意：Kubernetes从0开始计数，而ZooKeeper是从1开始计数。
查看FQDN（Fully Qualified Domain Name）：
$ for i in 0 1 2;do kubectl exec zk-$i-- hostname -f; done
zk-0.zk-hs.default.svc.cluster.local
zk-1.zk-hs.default.svc.cluster.local
zk-2.zk-hs.default.svc.cluster.local
ZooKeeper headless service为StatefulSet中的每个pod创建一个域名。
Kubernetes DNS中的A记录将FQDN解析为pod的IP地址。将来如果pod被重新调度或者升级，则A记录会放置新的IP地址，但是名字保持不变。
ZooKeeper使用一个
zoo.cfg
的配置文件（
/opt/zookeeper/conf/zoo.cfg
）。我们来查看该文件：
$ kubectl exec zk-0 --cat/opt/zookeeper/conf/zoo.cfg
#This file was autogenerated DO NOT EDIT
clientPort=2181
dataDir=/var/lib/zookeeper/data
dataLogDir=/var/lib/zookeeper/data/log
tickTime=2000
initLimit=10
syncLimit=5
maxClientCnxns=60
minSessionTimeout=4000
maxSessionTimeout=40000
autopurge.snapRetainCount=3
autopurge.purgeInteval=12
server.1=zk-0.zk-hs.default.svc.cluster.local:2888:3888
server.2=zk-1.zk-hs.default.svc.cluster.local:2888:3888
server.3=zk-2.zk-hs.default.svc.cluster.local:2888:3888
该文件是由
start-zookeeper
创建的：
......- sh
--c
- "start-zookeeper \
--servers=3 \
--data_dir=/var/lib/zookeeper/data \
--data_log_dir=/var/lib/zookeeper/data/log \
--conf_dir=/opt/zookeeper/conf \
......
测试
使用ZooKeeper的命令行工具
zkCli.sh
来测试。它可以：
- 创建znode
- 获取数据
- 监视znode变化
- 把数据放置于znode
- 为znode创建子节点
- 列出子节点
- 检查状态
- 删除znode
ZooKeeper里的znode既像文件（可以有内容），也像目录（可以有子znode）。
我们来把
world
写入
zk-0
的
/hello
里。
$ kubectl exec zk-0 zkCli.sh create /hello world
kubectl exec [POD][COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD]--[COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:35:34,023 [myid:]- INFO [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:35:34,027 [myid:]- INFO [main:Environment@100]- Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:35:34,027 [myid:]- INFO [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:35:34,028 [myid:]- INFO [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:35:34,028 [myid:]- INFO [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:35:34,029 [myid:]- INFO [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:35:34,029 [myid:]- INFO [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:35:34,029 [myid:]- INFO [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:35:34,029 [myid:]- INFO [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:35:34,029 [myid:]- INFO [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:35:34,029 [myid:]- INFO [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:35:34,029 [myid:]- INFO [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:35:34,030 [myid:]- INFO [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:35:34,030 [myid:]- INFO [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:35:34,030 [myid:]- INFO [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:35:34,031 [myid:]- INFO [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:35:34,051 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:35:34,096 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:35:34,114 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330000, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
Created /hello
然后从另一个pod读取该值：
$ kubectl exec zk-1 -- zkCli.sh get /hello
Connecting to localhost:2181
2024-02-06 08:36:34,736 [myid:]- INFO [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:36:34,738 [myid:]- INFO [main:Environment@100]- Client environment:host.name=zk-1.zk-hs.default.svc.cluster.local
2024-02-06 08:36:34,738 [myid:]- INFO [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:36:34,740 [myid:]- INFO [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:36:34,740 [myid:]- INFO [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:36:34,740 [myid:]- INFO [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:36:34,740 [myid:]- INFO [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:36:34,740 [myid:]- INFO [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:36:34,740 [myid:]- INFO [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:36:34,740 [myid:]- INFO [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:36:34,740 [myid:]- INFO [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:36:34,741 [myid:]- INFO [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:36:34,741 [myid:]- INFO [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:36:34,741 [myid:]- INFO [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:36:34,741 [myid:]- INFO [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:36:34,742 [myid:]- INFO [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:36:34,755 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:36:34,820 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:36:34,833 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x28d7d581b360000, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
world
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000002
mtime = Tue Feb 06 08:35:34 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
事实上，ZooKeeper ensemble里的每个服务器都可以读取
/hello
：
$ for i in 0 1 2;do kubectl exec zk-$i-- zkCli.sh get /hello | grep world; done
world
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000002
mtime = Tue Feb 06 08:35:34 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
world
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000002
mtime = Tue Feb 06 08:35:34 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
cZxid = 0x100000002
world
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000002
mtime = Tue Feb 06 08:35:34 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
容忍node故障
我们来看一下共识算法。删除一个服务器，然后尝试向ZooKeeper ensemble中写入数据，由于还有两个ZooKeeper服务器，所以写入操作还能继续工作。
$ kubectl delete --force=true --grace-period=0 pod zk-2 &
sleep 1; kubectl delete --force=true --grace-period=0 pod zk-2 &
sleep 1
kubectl exec zk-0 zkCli.sh set/hello world_should_work
sleep 1
kubectl exec zk-1 zkCli.sh get /hello
[1] 148438
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "zk-2" force deleted
[1]+ Done kubectl delete --force=true --grace-period=0 pod zk-2
[1] 148686
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "zk-2" force deleted
[1]+ Done kubectl delete --force=true --grace-period=0 pod zk-2
kubectl exec [POD][COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD]--[COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:45:21,133 [myid:]- INFO [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:45:21,135 [myid:]- INFO [main:Environment@100]- Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:45:21,135 [myid:]- INFO [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:45:21,137 [myid:]- INFO [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:45:21,137 [myid:]- INFO [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:45:21,137 [myid:]- INFO [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:45:21,137 [myid:]- INFO [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:45:21,138 [myid:]- INFO [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:45:21,138 [myid:]- INFO [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:45:21,138 [myid:]- INFO [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:45:21,140 [myid:]- INFO [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:45:21,140 [myid:]- INFO [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:45:21,140 [myid:]- INFO [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:45:21,140 [myid:]- INFO [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:45:21,140 [myid:]- INFO [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:45:21,141 [myid:]- INFO [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:45:21,157 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:45:21,201 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:45:21,206 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330003, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0
kubectl exec [POD][COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD]--[COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:45:22,821 [myid:]- INFO [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:45:22,824 [myid:]- INFO [main:Environment@100]- Client environment:host.name=zk-1.zk-hs.default.svc.cluster.local
2024-02-06 08:45:22,824 [myid:]- INFO [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:45:22,827 [myid:]- INFO [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:45:22,827 [myid:]- INFO [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:45:22,827 [myid:]- INFO [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:45:22,827 [myid:]- INFO [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:45:22,827 [myid:]- INFO [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:45:22,827 [myid:]- INFO [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:45:22,828 [myid:]- INFO [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:45:22,828 [myid:]- INFO [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:45:22,828 [myid:]- INFO [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:45:22,828 [myid:]- INFO [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:45:22,828 [myid:]- INFO [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:45:22,828 [myid:]- INFO [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:45:22,830 [myid:]- INFO [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:45:22,845 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:45:22,897 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:45:22,905 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x28d7d581b360005, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
cZxid = 0x100000002
world_should_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0
验证：
$ for i in 0 1 2;do kubectl exec zk-$i-- zkCli.sh get /hello | grep world; done
cZxid = 0x100000002
world_should_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0
cZxid = 0x100000002
world_should_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0
cZxid = 0x100000002
world_should_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0
写入操作还能工作，是因为ZooKeeper ensemble的quorum（法定人数），如果删除两个服务器，就没有quorum了。
$ kubectl delete --force=true --grace-period=0 pod zk-2 &
kubectl delete --force=true --grace-period=0 pod zk-1 &
sleep 1; kubectl delete --force=true --grace-period=0 pod zk-2 &
sleep 1; kubectl delete --force=true --grace-period=0 pod zk-1 &
sleep 1
kubectl exec zk-0 zkCli.sh set/hello world_should_not_work
sleep 1
kubectl exec zk-0 zkCli.sh get /hello
sleep 20 # If you are running manually use kubectl get pods to see status of pods restarting
kubectl exec zk-0 zkCli.sh get /hello
[1] 159326
[2] 159327
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "zk-2" force deleted
pod "zk-1" force deleted
[1]- Done kubectl delete --force=true --grace-period=0 pod zk-2
[2]+ Done kubectl delete --force=true --grace-period=0 pod zk-1
[1] 159519
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
Error from server (NotFound): pods "zk-2" not found
[1]+Exit 1 kubectl delete --force=true --grace-period=0 pod zk-2
[1] 159526
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "zk-1" force deleted
[1]+ Done kubectl delete --force=true --grace-period=0 pod zk-1
kubectl exec [POD][COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD]--[COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:51:54,920 [myid:]- INFO [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:51:54,923 [myid:]- INFO [main:Environment@100]- Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:51:54,923 [myid:]- INFO [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:51:54,925 [myid:]- INFO [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:51:54,925 [myid:]- INFO [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:51:54,925 [myid:]- INFO [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:51:54,925 [myid:]- INFO [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:51:54,925 [myid:]- INFO [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:51:54,925 [myid:]- INFO [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:51:54,926 [myid:]- INFO [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:51:54,926 [myid:]- INFO [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:51:54,926 [myid:]- INFO [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:51:54,926 [myid:]- INFO [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:51:54,926 [myid:]- INFO [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:51:54,926 [myid:]- INFO [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:51:54,929 [myid:]- INFO [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:51:54,949 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:51:55,025 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:51:55,036 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330005, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000022
mtime = Tue Feb 06 08:51:55 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 21
numChildren = 0
kubectl exec [POD][COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD]--[COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:51:56,664 [myid:]- INFO [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:51:56,666 [myid:]- INFO [main:Environment@100]- Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:51:56,666 [myid:]- INFO [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:51:56,668 [myid:]- INFO [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:51:56,668 [myid:]- INFO [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:51:56,668 [myid:]- INFO [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:51:56,668 [myid:]- INFO [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:51:56,668 [myid:]- INFO [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:51:56,668 [myid:]- INFO [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:51:56,668 [myid:]- INFO [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:51:56,668 [myid:]- INFO [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:51:56,668 [myid:]- INFO [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:51:56,668 [myid:]- INFO [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:51:56,669 [myid:]- INFO [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:51:56,669 [myid:]- INFO [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:51:56,670 [myid:]- INFO [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:51:56,690 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:51:56,748 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:51:56,755 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330006, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
world_should_not_work
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000022
mtime = Tue Feb 06 08:51:55 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 21
numChildren = 0
kubectl exec [POD][COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD]--[COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:52:17,354 [myid:]- INFO [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:52:17,355 [myid:]- INFO [main:Environment@100]- Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:52:17,356 [myid:]- INFO [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:52:17,357 [myid:]- INFO [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:52:17,357 [myid:]- INFO [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:52:17,357 [myid:]- INFO [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:52:17,357 [myid:]- INFO [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:52:17,358 [myid:]- INFO [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:52:17,358 [myid:]- INFO [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:52:17,358 [myid:]- INFO [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:52:17,358 [myid:]- INFO [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:52:17,358 [myid:]- INFO [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:52:17,358 [myid:]- INFO [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:52:17,358 [myid:]- INFO [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:52:17,358 [myid:]- INFO [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:52:17,359 [myid:]- INFO [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:52:17,374 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:52:17,436 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:52:17,444 [myid:]- INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330007, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
cZxid = 0x100000002
world_should_not_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000022
mtime = Tue Feb 06 08:51:55 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 21
numChildren = 0
可见，还是写入了。我测试了几次，都是这样。难道是因为pod恢复的太快了？
持久化存储
删除StatefulSet：
kubectl delete statefulset zk
验证：
$ kubectl get pod
No resources found in default namespace.
重建StatefulSet：
$ kubectl apply -f zookeeper2.yaml
service/zk-hs unchanged
service/zk-cs unchanged
poddisruptionbudget.policy/zk-pdb configured
statefulset.apps/zk created
$ kubectl exec zk-0 -- zkCli.sh get /hello
......
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
ccccc
......
注：
ccccc
是最近一次写入的数据。
可见，重建StatefulSet和其pod之后，原来的数据还在。
当ZooKeeper的StatefulSet的pod被重新调度或者升级时，会将PV mount到ZooKeeper服务器的数据目录。持久化数据仍然保存在那里。对于Cassandra或者Consul或者etcd或其它任何数据库也都一样。
参考
https://github.com/cloudurable/kube-zookeeper-statefulsets/wiki/Tutorial-Part-1:--Managing-Kubernetes-StatefulSets-using-ZooKeeper-and-Minikubehttps://kubernetes.io/docs/tutorials/stateful-application/zookeeper
本文转载自: https://blog.csdn.net/duke_ding2/article/details/136057094
版权归原作者 蓝黑2020 所有， 如有侵权，请联系我们删除。
版权归原作者 蓝黑2020 所有， 如有侵权，请联系我们删除。