Kubernetes的有状态应用示例：ZooKeeper

文章目录

环境

RHEL 9.3
Docker Community 24.0.7
minikube v1.32.0

ZooKeeper简介

Apache ZooKeeper是一个分布式的开源协调服务，用于分布式系统。ZooKeeper允许你读、写数据以及发现数据更新。数据按层次结构组织在文件系统中，并复制到ensemble（ZooKeeper服务器集合）中所有的ZooKeeper服务器。对数据的所有操作都是原子的和顺序一致的。ZooKeeper通过Zab共识协议在ensemble的所有服务器之间复制状态机，来确保这个特性。

Ensemble使用Zab协议选举一个leader，在选举出leader前不能写入数据。选举出leader后，ensemble使用Zab来确保所有写操作被复制到一个quorum，然后这些写操作才会被确认并对客户端可见。如果没有遵照加权quorums，quorum是包含当前leader的ensemble的主要组件。例如，如果ensemble有3个服务器，一个包含leader和另一个服务器的组件构成一个quorum。如果ensemble不能达成一个quorum，数据将不能被写入。

ZooKeeper在内存中保存它们的整个状态机，并把每个改变都写入一个持久的WAL（Write Ahead Log）。当一个服务器宕机时，它能够通过回放WAL恢复之前的状态。为了防止WAL无限制的增长，ZooKeeper服务器会定期的将内存状态快照保存到存储介质。这些快照能够直接加载到内存，快照之前的所有WAL条目都可以被丢弃。

准备

清理环境：

minikube delete --all

重启电脑。

启动minikube：

minikube start

确认环境干净：

$ kubectl get all
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   11m

$ kubectl get pvc
No resources found in default namespace.

$ kubectl get pv
No resources found

创建文件

zookeeper.yaml

如下：

apiVersion: v1
kind: Service
metadata:name: zk-hs
  labels:app: zk
spec:ports:-port:2888name: server
  -port:3888name: leader-election
  clusterIP: None
  selector:app: zk
---apiVersion: v1
kind: Service
metadata:name: zk-cs
  labels:app: zk
spec:ports:-port:2181name: client
  selector:app: zk
---apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:name: zk-pdb
spec:selector:matchLabels:app: zk
  maxUnavailable:1---apiVersion: apps/v1
kind: StatefulSet
metadata:name: zk
spec:selector:matchLabels:app: zk
  serviceName: zk-hs
  replicas:3updateStrategy:type: RollingUpdate
  podManagementPolicy: OrderedReady
  template:metadata:labels:app: zk
    spec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:-labelSelector:matchExpressions:-key:"app"operator: In
                    values:- zk
              topologyKey:"kubernetes.io/hostname"containers:-name: kubernetes-zookeeper
        imagePullPolicy: Always
        # image: "registry.k8s.io/kubernetes-zookeeper:1.0-3.4.10"image:"docker.io/kaiding1/kubernetes-zookeeper:1.0-3.4.10"resources:requests:memory:"1Gi"cpu:"0.5"ports:-containerPort:2181name: client
        -containerPort:2888name: server
        -containerPort:3888name: leader-election
        command:- sh
        --c
        - "start-zookeeper \
          --servers=3 \
          --data_dir=/var/lib/zookeeper/data \
          --data_log_dir=/var/lib/zookeeper/data/log \
          --conf_dir=/opt/zookeeper/conf \
          --client_port=2181 \
          --election_port=3888 \
          --server_port=2888 \
          --tick_time=2000 \
          --init_limit=10 \
          --sync_limit=5 \
          --heap=512M \
          --max_client_cnxns=60 \
          --snap_retain_count=3 \
          --purge_interval=12 \
          --max_session_timeout=40000 \
          --min_session_timeout=4000 \
          --log_level=INFO"
        readinessProbe:exec:command:- sh
            --c
            -"zookeeper-ready 2181"initialDelaySeconds:10timeoutSeconds:5livenessProbe:exec:command:- sh
            --c
            -"zookeeper-ready 2181"initialDelaySeconds:10timeoutSeconds:5volumeMounts:-name: datadir
          mountPath: /var/lib/zookeeper
      securityContext:runAsUser:1000fsGroup:1000volumeClaimTemplates:-metadata:name: datadir
    spec:accessModes:["ReadWriteOnce"]resources:requests:storage: 10Gi

注：因为访问不了

registry.k8s.io

，所以事先把image pull下来，并push到了可访问的位置。

部署

尝试1

$ kubectl apply -f zookeeper.yaml
service/zk-hs created
service/zk-cs created
poddisruptionbudget.policy/zk-pdb created
statefulset.apps/zk created

检查pod：

$ kubectl get pod
NAME   READY   STATUS    RESTARTS   AGE
zk-0   1/1     Running   0          57s
zk-1   0/1     Pending   0          35s

进入

zk-0

pod：

kubectl exec -it zk-0 -- bash

查看当前身份以及

/var/lib/zookeeper

zookeeper@zk-0:/$ whoami
zookeeper

zookeeper@zk-0:/$ ls-l /var/lib
total 0
drwxr-xr-x. 1 root root  42 Jun 13  2017 apt
......
drwxrwxrwx. 3 root root  18 Feb  6 07:13 zookeeper

zookeeper@zk-0:/$ ls-l /var/lib/zookeeper/
total 0
drwxr-xr-x. 4 zookeeper zookeeper 46 Feb  6 07:13 data

可见，

/var/lib/zookeeper

目录是

，所以

zookeeper

能在此创建目录。

退出容器，再次查看pod：

$ kubectl get pod
NAME   READY   STATUS    RESTARTS   AGE
zk-0   1/1     Running   0          7m37s
zk-1   0/1     Pending   0          7m15s

可见，

zk-1

始终处于

Pending

状态。

$ kubectl describe pod zk-1
......
Events:
  Type     Reason            Age                   From               Message
  -------------------------
  Warning  FailedScheduling  8m4s                  default-scheduler  0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
  Warning  FailedScheduling  2m39s (x2 over 8m3s)  default-scheduler  0/1 nodes are available: 1 node(s) didn't match pod anti-affinity rules. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod..

可见，在affinity规则下，没有符合条件的node。

尝试2

清理环境：

$ kubectl delete -f zookeeper.yaml 
service "zk-hs" deleted
service "zk-cs" deleted
poddisruptionbudget.policy "zk-pdb" deleted
statefulset.apps "zk" deleted

$ kubectl get pvc
NAME           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
datadir-zk-0   Bound    pvc-f4a42476-b3e7-48ca-ad8d-b8f6ec278f1e   10Gi       RWO            standard       15m
datadir-zk-1   Bound    pvc-924378b6-5810-4ebc-86a7-90037221381f   10Gi       RWO            standard       14m

$ kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                  STORAGECLASS   REASON   AGE
pvc-924378b6-5810-4ebc-86a7-90037221381f   10Gi       RWO            Delete           Bound    default/datadir-zk-1   standard                14m
pvc-f4a42476-b3e7-48ca-ad8d-b8f6ec278f1e   10Gi       RWO            Delete           Bound    default/datadir-zk-0   standard                15m

$ kubectl delete pvc datadir-zk-0 datadir-zk-1
persistentvolumeclaim "datadir-zk-0" deleted
persistentvolumeclaim "datadir-zk-1" deleted

修改

zookeeper.yaml

，把

requiredDuringSchedulingIgnoredDuringExecution

改为

preferredDuringSchedulingIgnoredDuringExecution

。

......spec:affinity:podAntiAffinity:preferredDuringSchedulingIgnoredDuringExecution:-weight:100podAffinityTerm:labelSelector:matchExpressions:-key:"app"operator: In
                      values:- zookeeper
                topologyKey:"kubernetes.io/hostname"......

$ kubectl apply -f zookeeper2.yaml
service/zk-hs created
service/zk-cs created
poddisruptionbudget.policy/zk-pdb created
statefulset.apps/zk created

$ kubectl get pod
NAME   READY   STATUS    RESTARTS   AGE
zk-0   1/1     Running   0          2m1s
zk-1   1/1     Running   0          99s
zk-2   1/1     Running   0          77s

$ kubectl get sts
NAME   READY   AGE
zk     3/3     6m31s

注：如果又遇到pod

Pending

问题，有以下错误：

Events:
  Type     Reason            Age        From               Message
  -------------------------
  Warning  FailedScheduling  <unknown>  default-scheduler  0/1 nodes are available: 1 Insufficient memory.
  Warning  FailedScheduling  <unknown>  default-scheduler  0/1 nodes are available: 1 Insufficient memory.

则需要先

minikube delete

，然后在启动minikube时，增大内存，比如：

minikube start--memory='7500mb'

查看存储：

$ kubectl get pvc
NAME           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
datadir-zk-0   Bound    pvc-7b501343-085f-48d8-b88a-ade5686b26c9   10Gi       RWO            standard       7m18s
datadir-zk-1   Bound    pvc-4af3ab29-a3c8-4b26-addf-40656e998021   10Gi       RWO            standard       6m56s
datadir-zk-2   Bound    pvc-60869d4a-389d-4f86-8446-4fda9858fe40   10Gi       RWO            standard       6m34s

$ kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                  STORAGECLASS   REASON   AGE
pvc-4af3ab29-a3c8-4b26-addf-40656e998021   10Gi       RWO            Delete           Bound    default/datadir-zk-1   standard                6m58s
pvc-60869d4a-389d-4f86-8446-4fda9858fe40   10Gi       RWO            Delete           Bound    default/datadir-zk-2   standard                6m36s
pvc-7b501343-085f-48d8-b88a-ade5686b26c9   10Gi       RWO            Delete           Bound    default/datadir-zk-0   standard                7m20s

验证

进入容器：

kubectl exec -it zk-0 -- bash

检查连通性：

zookeeper@zk-0:/$ echo"Are you ok? $(echo ruok | nc 127.0.0.1 2181)"
Are you ok? imok

检查模式：

zookeeper@zk-0:/$ echo srvr | nc localhost 2181
Zookeeper version: 3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
Latency min/avg/max: 0/0/0
Received: 123
Sent: 122
Connections: 1
Outstanding: 0
Zxid: 0x0
Mode: follower
Node count: 4

可见，pod

zk-0

是一个follower。

退出容器，用同样的方法检查另外两个pod：

zookeeper@zk-1:/$ echo srvr | nc localhost 2181 | grep -i mode
Mode: leader

zookeeper@zk-2:/$ echo srvr | nc localhost 2181 | grep -i mode
Mode: follower

在leader pod里：

zookeeper@zk-1:/$ echo dump | nc localhost 2181
SessionTracker dump:
Session Sets (0):
ephemeral nodes dump:
Sessions with Ephemerals (0):

zookeeper@zk-1:/$ echo stat | nc localhost 2181
Zookeeper version: 3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
Clients:
 /127.0.0.1:60780[0](queued=0,recved=1,sent=0)
Latency min/avg/max: 0/0/0
Received: 221
Sent: 220
Connections: 1
Outstanding: 0
Zxid: 0x100000000
Mode: leader
Node count: 4

zookeeper@zk-1:/$ echo envi | nc localhost 2181
Environment:
zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
host.name=zk-1.zk-hs.default.svc.cluster.local
java.version=1.8.0_131
java.vendor=Oracle Corporation
java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
java.io.tmpdir=/tmp
java.compiler=<NA>
os.name=Linux
os.arch=amd64
os.version=5.14.0-362.18.1.el9_3.x86_64
user.name=zookeeper
user.home=/home/zookeeper
user.dir=/

zookeeper@zk-1:/$ echo conf | nc localhost 2181
clientPort=2181
dataDir=/var/lib/zookeeper/data/version-2
dataLogDir=/var/lib/zookeeper/data/log/version-2
tickTime=2000
maxClientCnxns=60
minSessionTimeout=4000
maxSessionTimeout=40000
serverId=2
initLimit=10
syncLimit=5
electionAlg=3
electionPort=3888
quorumPort=2888
peerType=0

zookeeper@zk-1:/$ echo mntr | nc localhost 2181
zk_version    3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
zk_avg_latency    0
zk_max_latency    0
zk_min_latency    0
zk_packets_received    242
zk_packets_sent    241
zk_num_alive_connections    1
zk_outstanding_requests    0
zk_server_state    leader
zk_znode_count    4
zk_watch_count    0
zk_ephemerals_count    0
zk_approximate_data_size    27
zk_open_file_descriptor_count    41
zk_max_file_descriptor_count    1048576
zk_followers    2
zk_synced_followers    2
zk_pending_syncs    0

事实上，在spec中，使用了

zookeeper-ready 2181

来作为

readinessProbe

和

livenessProbe

：

......readinessProbe:exec:command:- sh
            --c
            -"zookeeper-ready 2181"initialDelaySeconds:10timeoutSeconds:5livenessProbe:exec:command:- sh
            --c
            -"zookeeper-ready 2181"initialDelaySeconds:10timeoutSeconds:5......

查看

zookeeper-ready

：

zookeeper@zk-1:/$ which zookeeper-ready
/usr/bin/zookeeper-ready

zookeeper@zk-1:/$ cat/usr/bin/zookeeper-ready
#!/usr/bin/env bash# Copyright 2017 The Kubernetes Authors.## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at##     http://www.apache.org/licenses/LICENSE-2.0## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License.# zkOk.sh uses the ruok ZooKeeper four letter work to determine if the instance# is health. The $? variable will be set to 0 if server responds that it is # healthy, or 1 if the server fails to respond.
OK=$(echo ruok | nc 127.0.0.1 $1)if["$OK" == "imok"]; then
    exit 0
elseexit 1
fi

可见，它就是用上面的方法来检测连通性的。

深入了解ZooKeeper

leader和follower

每个ZooKeeper ensemble里的服务器节点都有一个唯一的ID与其网络地址关联。每个node都知道该ID。

hostname：

$ for i in 0 1 2;do kubectl exec zk-$i-- hostname; done
zk-0
zk-1
zk-2

ZooKeeper node在数据目录的

myid

文件中存储其服务器ID（本例中数据目录是

/var/lib/zookeeper

）。

$ for i in 0 1 2;doecho"myid zk-$i"; kubectl exec zk-$i--cat/var/lib/zookeeper/data/myid; done
myid zk-0
1
myid zk-1
2
myid zk-2
3

注意：Kubernetes从0开始计数，而ZooKeeper是从1开始计数。

查看FQDN（Fully Qualified Domain Name）：

$ for i in 0 1 2;do kubectl exec zk-$i-- hostname -f; done
zk-0.zk-hs.default.svc.cluster.local
zk-1.zk-hs.default.svc.cluster.local
zk-2.zk-hs.default.svc.cluster.local

ZooKeeper headless service为StatefulSet中的每个pod创建一个域名。

Kubernetes DNS中的A记录将FQDN解析为pod的IP地址。将来如果pod被重新调度或者升级，则A记录会放置新的IP地址，但是名字保持不变。

ZooKeeper使用一个

zoo.cfg

的配置文件（

/opt/zookeeper/conf/zoo.cfg

）。我们来查看该文件：

$ kubectl exec zk-0 --cat/opt/zookeeper/conf/zoo.cfg
#This file was autogenerated DO NOT EDIT
clientPort=2181
dataDir=/var/lib/zookeeper/data
dataLogDir=/var/lib/zookeeper/data/log
tickTime=2000
initLimit=10
syncLimit=5
maxClientCnxns=60
minSessionTimeout=4000
maxSessionTimeout=40000
autopurge.snapRetainCount=3
autopurge.purgeInteval=12
server.1=zk-0.zk-hs.default.svc.cluster.local:2888:3888
server.2=zk-1.zk-hs.default.svc.cluster.local:2888:3888
server.3=zk-2.zk-hs.default.svc.cluster.local:2888:3888

该文件是由

start-zookeeper

创建的：

......- sh
        --c
        - "start-zookeeper \
          --servers=3 \
          --data_dir=/var/lib/zookeeper/data \
          --data_log_dir=/var/lib/zookeeper/data/log \
          --conf_dir=/opt/zookeeper/conf \
......

测试

使用ZooKeeper的命令行工具

 zkCli.sh

来测试。它可以：

创建znode
获取数据
监视znode变化
把数据放置于znode
为znode创建子节点
列出子节点
检查状态
删除znode

ZooKeeper里的znode既像文件（可以有内容），也像目录（可以有子znode）。

我们来把

world

写入

zk-0

的

/hello

里。

$ kubectl exec zk-0 zkCli.sh create /hello world
kubectl exec [POD][COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD]--[COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:35:34,023 [myid:]- INFO  [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:35:34,027 [myid:]- INFO  [main:Environment@100]- Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:35:34,027 [myid:]- INFO  [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:35:34,028 [myid:]- INFO  [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:35:34,028 [myid:]- INFO  [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:35:34,029 [myid:]- INFO  [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:35:34,029 [myid:]- INFO  [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:35:34,029 [myid:]- INFO  [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:35:34,029 [myid:]- INFO  [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:35:34,029 [myid:]- INFO  [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:35:34,029 [myid:]- INFO  [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:35:34,029 [myid:]- INFO  [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:35:34,030 [myid:]- INFO  [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:35:34,030 [myid:]- INFO  [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:35:34,030 [myid:]- INFO  [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:35:34,031 [myid:]- INFO  [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:35:34,051 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:35:34,096 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:35:34,114 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330000, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
Created /hello

然后从另一个pod读取该值：

$ kubectl exec zk-1 -- zkCli.sh get /hello
Connecting to localhost:2181
2024-02-06 08:36:34,736 [myid:]- INFO  [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:36:34,738 [myid:]- INFO  [main:Environment@100]- Client environment:host.name=zk-1.zk-hs.default.svc.cluster.local
2024-02-06 08:36:34,738 [myid:]- INFO  [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:36:34,740 [myid:]- INFO  [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:36:34,740 [myid:]- INFO  [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:36:34,740 [myid:]- INFO  [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:36:34,740 [myid:]- INFO  [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:36:34,740 [myid:]- INFO  [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:36:34,740 [myid:]- INFO  [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:36:34,740 [myid:]- INFO  [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:36:34,740 [myid:]- INFO  [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:36:34,741 [myid:]- INFO  [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:36:34,741 [myid:]- INFO  [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:36:34,741 [myid:]- INFO  [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:36:34,741 [myid:]- INFO  [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:36:34,742 [myid:]- INFO  [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:36:34,755 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:36:34,820 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:36:34,833 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x28d7d581b360000, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
world
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000002
mtime = Tue Feb 06 08:35:34 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0

事实上，ZooKeeper ensemble里的每个服务器都可以读取

/hello

：

$ for i in 0 1 2;do kubectl exec zk-$i-- zkCli.sh get /hello | grep world; done
world
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000002
mtime = Tue Feb 06 08:35:34 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
world
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000002
mtime = Tue Feb 06 08:35:34 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
cZxid = 0x100000002
world
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000002
mtime = Tue Feb 06 08:35:34 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0

容忍node故障

我们来看一下共识算法。删除一个服务器，然后尝试向ZooKeeper ensemble中写入数据，由于还有两个ZooKeeper服务器，所以写入操作还能继续工作。

$ kubectl delete --force=true --grace-period=0 pod zk-2  &
sleep 1; kubectl delete --force=true --grace-period=0 pod zk-2  &
sleep 1
kubectl exec zk-0 zkCli.sh set/hello world_should_work
sleep 1
kubectl exec zk-1 zkCli.sh get /hello
[1] 148438
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "zk-2" force deleted
[1]+  Done                    kubectl delete --force=true --grace-period=0 pod zk-2
[1] 148686
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "zk-2" force deleted
[1]+  Done                    kubectl delete --force=true --grace-period=0 pod zk-2
kubectl exec [POD][COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD]--[COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:45:21,133 [myid:]- INFO  [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:45:21,135 [myid:]- INFO  [main:Environment@100]- Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:45:21,135 [myid:]- INFO  [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:45:21,137 [myid:]- INFO  [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:45:21,137 [myid:]- INFO  [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:45:21,137 [myid:]- INFO  [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:45:21,137 [myid:]- INFO  [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:45:21,138 [myid:]- INFO  [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:45:21,138 [myid:]- INFO  [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:45:21,138 [myid:]- INFO  [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:45:21,140 [myid:]- INFO  [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:45:21,140 [myid:]- INFO  [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:45:21,140 [myid:]- INFO  [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:45:21,140 [myid:]- INFO  [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:45:21,140 [myid:]- INFO  [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:45:21,141 [myid:]- INFO  [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:45:21,157 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:45:21,201 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:45:21,206 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330003, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0
kubectl exec [POD][COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD]--[COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:45:22,821 [myid:]- INFO  [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:45:22,824 [myid:]- INFO  [main:Environment@100]- Client environment:host.name=zk-1.zk-hs.default.svc.cluster.local
2024-02-06 08:45:22,824 [myid:]- INFO  [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:45:22,827 [myid:]- INFO  [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:45:22,827 [myid:]- INFO  [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:45:22,827 [myid:]- INFO  [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:45:22,827 [myid:]- INFO  [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:45:22,827 [myid:]- INFO  [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:45:22,827 [myid:]- INFO  [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:45:22,828 [myid:]- INFO  [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:45:22,828 [myid:]- INFO  [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:45:22,828 [myid:]- INFO  [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:45:22,828 [myid:]- INFO  [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:45:22,828 [myid:]- INFO  [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:45:22,828 [myid:]- INFO  [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:45:22,830 [myid:]- INFO  [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:45:22,845 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:45:22,897 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:45:22,905 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x28d7d581b360005, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
cZxid = 0x100000002
world_should_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0

验证：

$ for i in 0 1 2;do kubectl exec zk-$i-- zkCli.sh get /hello | grep world; done
cZxid = 0x100000002
world_should_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0
cZxid = 0x100000002
world_should_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0
cZxid = 0x100000002
world_should_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0

写入操作还能工作，是因为ZooKeeper ensemble的quorum（法定人数），如果删除两个服务器，就没有quorum了。

$ kubectl delete --force=true --grace-period=0 pod zk-2  &
kubectl delete --force=true --grace-period=0 pod zk-1  &
sleep 1; kubectl delete --force=true --grace-period=0 pod zk-2  &
sleep 1; kubectl delete --force=true --grace-period=0 pod zk-1  &
sleep 1
kubectl exec zk-0 zkCli.sh set/hello world_should_not_work
sleep 1
kubectl exec zk-0 zkCli.sh get /hello
sleep 20 # If you are running manually use kubectl get pods to see status of pods restarting
kubectl exec zk-0 zkCli.sh get /hello
[1] 159326
[2] 159327
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "zk-2" force deleted
pod "zk-1" force deleted
[1]-  Done                    kubectl delete --force=true --grace-period=0 pod zk-2
[2]+  Done                    kubectl delete --force=true --grace-period=0 pod zk-1
[1] 159519
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
Error from server (NotFound): pods "zk-2" not found
[1]+Exit 1                  kubectl delete --force=true --grace-period=0 pod zk-2
[1] 159526
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "zk-1" force deleted
[1]+  Done                    kubectl delete --force=true --grace-period=0 pod zk-1
kubectl exec [POD][COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD]--[COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:51:54,920 [myid:]- INFO  [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:51:54,923 [myid:]- INFO  [main:Environment@100]- Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:51:54,923 [myid:]- INFO  [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:51:54,925 [myid:]- INFO  [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:51:54,925 [myid:]- INFO  [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:51:54,925 [myid:]- INFO  [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:51:54,925 [myid:]- INFO  [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:51:54,925 [myid:]- INFO  [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:51:54,925 [myid:]- INFO  [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:51:54,926 [myid:]- INFO  [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:51:54,926 [myid:]- INFO  [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:51:54,926 [myid:]- INFO  [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:51:54,926 [myid:]- INFO  [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:51:54,926 [myid:]- INFO  [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:51:54,926 [myid:]- INFO  [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:51:54,929 [myid:]- INFO  [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:51:54,949 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:51:55,025 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:51:55,036 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330005, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000022
mtime = Tue Feb 06 08:51:55 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 21
numChildren = 0
kubectl exec [POD][COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD]--[COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:51:56,664 [myid:]- INFO  [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:51:56,666 [myid:]- INFO  [main:Environment@100]- Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:51:56,666 [myid:]- INFO  [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:51:56,668 [myid:]- INFO  [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:51:56,668 [myid:]- INFO  [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:51:56,668 [myid:]- INFO  [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:51:56,668 [myid:]- INFO  [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:51:56,668 [myid:]- INFO  [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:51:56,668 [myid:]- INFO  [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:51:56,668 [myid:]- INFO  [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:51:56,668 [myid:]- INFO  [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:51:56,668 [myid:]- INFO  [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:51:56,668 [myid:]- INFO  [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:51:56,669 [myid:]- INFO  [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:51:56,669 [myid:]- INFO  [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:51:56,670 [myid:]- INFO  [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:51:56,690 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:51:56,748 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:51:56,755 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330006, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
world_should_not_work
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000022
mtime = Tue Feb 06 08:51:55 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 21
numChildren = 0
kubectl exec [POD][COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD]--[COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:52:17,354 [myid:]- INFO  [main:Environment@100]- Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:52:17,355 [myid:]- INFO  [main:Environment@100]- Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:52:17,356 [myid:]- INFO  [main:Environment@100]- Client environment:java.version=1.8.0_131
2024-02-06 08:52:17,357 [myid:]- INFO  [main:Environment@100]- Client environment:java.vendor=Oracle Corporation
2024-02-06 08:52:17,357 [myid:]- INFO  [main:Environment@100]- Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:52:17,357 [myid:]- INFO  [main:Environment@100]- Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:52:17,357 [myid:]- INFO  [main:Environment@100]- Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:52:17,358 [myid:]- INFO  [main:Environment@100]- Client environment:java.io.tmpdir=/tmp
2024-02-06 08:52:17,358 [myid:]- INFO  [main:Environment@100]- Client environment:java.compiler=<NA>
2024-02-06 08:52:17,358 [myid:]- INFO  [main:Environment@100]- Client environment:os.name=Linux
2024-02-06 08:52:17,358 [myid:]- INFO  [main:Environment@100]- Client environment:os.arch=amd64
2024-02-06 08:52:17,358 [myid:]- INFO  [main:Environment@100]- Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:52:17,358 [myid:]- INFO  [main:Environment@100]- Client environment:user.name=zookeeper
2024-02-06 08:52:17,358 [myid:]- INFO  [main:Environment@100]- Client environment:user.home=/home/zookeeper
2024-02-06 08:52:17,358 [myid:]- INFO  [main:Environment@100]- Client environment:user.dir=/
2024-02-06 08:52:17,359 [myid:]- INFO  [main:ZooKeeper@438]- Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:52:17,374 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032]- Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:52:17,436 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876]- Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:52:17,444 [myid:]- INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299]- Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330007, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
cZxid = 0x100000002
world_should_not_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000022
mtime = Tue Feb 06 08:51:55 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 21
numChildren = 0

可见，还是写入了。我测试了几次，都是这样。难道是因为pod恢复的太快了？

持久化存储

删除StatefulSet：

kubectl delete statefulset zk

验证：

$ kubectl get pod
No resources found in default namespace.

重建StatefulSet：

$ kubectl apply -f zookeeper2.yaml
service/zk-hs unchanged
service/zk-cs unchanged
poddisruptionbudget.policy/zk-pdb configured
statefulset.apps/zk created

$ kubectl exec zk-0 -- zkCli.sh get /hello
......
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
ccccc
......

注：

ccccc

是最近一次写入的数据。

可见，重建StatefulSet和其pod之后，原来的数据还在。

当ZooKeeper的StatefulSet的pod被重新调度或者升级时，会将PV mount到ZooKeeper服务器的数据目录。持久化数据仍然保存在那里。对于Cassandra或者Consul或者etcd或其它任何数据库也都一样。

参考

https://github.com/cloudurable/kube-zookeeper-statefulsets/wiki/Tutorial-Part-1:--Managing-Kubernetes-StatefulSets-using-ZooKeeper-and-Minikube
https://kubernetes.io/docs/tutorials/stateful-application/zookeeper

标签： kubernetes zookeeper statefulset

本文转载自: https://blog.csdn.net/duke_ding2/article/details/136057094
版权归原作者 蓝黑2020 所有，如有侵权，请联系我们删除。

Kubernetes的有状态应用示例：ZooKeeper

文章目录

环境

ZooKeeper简介

准备

部署

尝试1

尝试2

验证

深入了解ZooKeeper

leader和follower

测试

容忍node故障

持久化存储

参考

发表评论

“Kubernetes的有状态应用示例：ZooKeeper”的评论:

关于作者

overfit同步小助手

相关阅读

文章导航