文章目录
- Kubernetes
# Kubernetes
https://iothub.org.cn/docs/kubernetes/
https://iothub.org.cn/docs/kubernetes/middleware/rabbitmq/
一、概述
1.Operator
# awesome-operators
https://github.com/operator-framework/awesome-operators
# 安装# RabbitMQ #1 (Official) rabbitmq/cluster-operator
https://github.com/rabbitmq/cluster-operator
App NameGithubDescriptionRabbitMQ #1 (Official)rabbitmq/cluster-operatorRabbitMQ cluster operator for Kubernetes.RabbitMQ #2skylt/rabbitmq-operatorRabbitMQ operator for Kubernetes.
2.Helm
Bitnami
https://bitnami.com/
https://github.com/bitnami
https://bitnami.com/stacks
[root@k8s-master redis-cluster-operator]# helm search repo rabbitmq
NAME CHART VERSION APP VERSION DESCRIPTION
bitnami/rabbitmq 8.23.0 3.9.7 Open source message broker software that implem...
bitnami/rabbitmq-cluster-operator 0.1.6 1.9.0 The RabbitMQ Cluster Kubernetes Operator automa...
......
二、基础
1.Operator
# 安装# RabbitMQ #1 (Official) rabbitmq/cluster-operator
https://github.com/rabbitmq/cluster-operator
#RabbitMQ Cluster Kubernetes Operator Quickstart
https://www.rabbitmq.com/kubernetes/operator/quickstart-operator.html
# Using RabbitMQ Cluster Kubernetes Operator
https://www.rabbitmq.com/kubernetes/operator/using-operator.html
# 参考# Operator overview
https://www.rabbitmq.com/kubernetes/operator/operator-overview.html
# Deploying an operator
https://www.rabbitmq.com/kubernetes/operator/install-operator.html
# Deploying a RabbitMQ cluster
https://www.rabbitmq.com/kubernetes/operator/using-operator.html
# Monitoring the cluster
https://www.rabbitmq.com/kubernetes/operator/operator-monitoring.html
# Troubleshooting operator deployments
https://www.rabbitmq.com/kubernetes/operator/troubleshooting-operator.html
下载rabbitmq/cluster-operator
# git git clone https://github.com/rabbitmq/cluster-operator.git[root@k8s-master rabbitmq]# git clone https://github.com/rabbitmq/cluster-operator.git
Cloning into 'cluster-operator'...
remote: Enumerating objects: 11892, done.
remote: Counting objects: 100% (2279/2279), done.
remote: Compressing objects: 100% (976/976), done.
remote: Total 11892(delta 1438), reused 1976(delta 1240), pack-reused 9613
Receiving objects: 100% (11892/11892), 11.95 MiB |1.66 MiB/s, done.
Resolving deltas: 100% (7740/7740), done.
1.1.Quickstart(单实例)
#RabbitMQ Cluster Kubernetes Operator Quickstart
https://www.rabbitmq.com/kubernetes/operator/quickstart-operator.html
Quickstart Steps
This guide goes through the following steps:
Install the RabbitMQ Cluster Operator
Deploy a RabbitMQ Cluster using the Operator
View RabbitMQ Logs
Access the RabbitMQ Management UI
Attach a Workload to the Cluster
Next Steps
1.创建Operator
#kubectl apply -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml[root@k8s-master docs]# kubectl apply -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml
namespace/rabbitmq-system created
customresourcedefinition.apiextensions.k8s.io/rabbitmqclusters.rabbitmq.com created
serviceaccount/rabbitmq-cluster-operator created
role.rbac.authorization.k8s.io/rabbitmq-cluster-leader-election-role created
clusterrole.rbac.authorization.k8s.io/rabbitmq-cluster-operator-role created
rolebinding.rbac.authorization.k8s.io/rabbitmq-cluster-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/rabbitmq-cluster-operator-rolebinding created
deployment.apps/rabbitmq-cluster-operator created
# 查看# 会创建一个新的名称空间:rabbitmq-system[root@k8s-master docs]# kubectl get all -n rabbitmq-system
NAME READY STATUS RESTARTS AGE
pod/rabbitmq-cluster-operator-7cbf865f89-5c4wk 1/1 Running 0 81s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/rabbitmq-cluster-operator 1/1 11 81s
NAME DESIRED CURRENT READY AGE
replicaset.apps/rabbitmq-cluster-operator-7cbf865f89 111 81s
2.使用cluster-operator创建RabbitMQ
简单的yaml文件下载地址:https://github.com/rabbitmq/cluster-operator/tree/main/docs/examples/hello-world
yaml文件名:rabbitmq.yaml
这是最简单的RabbitmqCluster定义。唯一显式指定的属性是集群的名称。其他一切都将根据集群运营商的默认值进行配置。
examples文件夹还有许多其他引用,比如用TLS、mTLS创建RabbitMQ集群,用生产默认值设置集群,添加社区插件等等。下载地址:https://github.com/rabbitmq/cluster-operator/tree/main/docs/examples/
# rabbitmq.yamlapiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:name: hello-world
# kubectl apply -f https://raw.githubusercontent.com/rabbitmq/cluster-operator/main/docs/examples/hello-world/rabbitmq.yaml[root@k8s-master docs]# kubectl apply -f https://raw.githubusercontent.com/rabbitmq/cluster-operator/main/docs/examples/hello-world/rabbitmq.yaml
rabbitmqcluster.rabbitmq.com/hello-world created
[root@k8s-master docs]# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/hello-world-server-0 0/1 Pending 0 74s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/hello-world ClusterIP 10.100.123.79 <none>15672/TCP,15692/TCP,5672/TCP 74s
service/hello-world-nodes ClusterIP None <none>4369/TCP,25672/TCP 74s
NAME READY AGE
statefulset.apps/hello-world-server 0/1 74s
NAME ALLREPLICASREADY RECONCILESUCCESS AGE
rabbitmqcluster.rabbitmq.com/hello-world False Unknown 74s
[root@k8s-master rabbitmq]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistence-hello-world-server-0 Pending 16m
# 查找Pending愿意,没有存储[root@k8s-master docs]# kubectl describe pod hello-world-server-0
Name: hello-world-server-0
Namespace: default
Priority: 0
Node: <none>
Labels: app.kubernetes.io/component=rabbitmq
app.kubernetes.io/name=hello-world
app.kubernetes.io/part-of=rabbitmq
controller-revision-hash=hello-world-server-6669f4bc8f
statefulset.kubernetes.io/pod-name=hello-world-server-0
Annotations: prometheus.io/port: 15692
prometheus.io/scrape: true
Status: Pending
IP:
IPs: <none>
Controlled By: StatefulSet/hello-world-server
Init Containers:
setup-container:
Image: rabbitmq:3.8.21-management
Port: <none>
Host Port: <none>
Command:
sh-ccp /tmp/erlang-cookie-secret/.erlang.cookie /var/lib/rabbitmq/.erlang.cookie &&chmod600 /var/lib/rabbitmq/.erlang.cookie ;cp /tmp/rabbitmq-plugins/enabled_plugins /operator/enabled_plugins ;echo'[default]'> /var/lib/rabbitmq/.rabbitmqadmin.conf &&sed-e's/default_user/username/'-e's/default_pass/password/' /tmp/default_user.conf >> /var/lib/rabbitmq/.rabbitmqadmin.conf &&chmod600 /var/lib/rabbitmq/.rabbitmqadmin.conf
Limits:
cpu: 100m
memory: 500Mi
Requests:
cpu: 100m
memory: 500Mi
Environment: <none>
Mounts:
/operator from rabbitmq-plugins (rw)
/tmp/default_user.conf from rabbitmq-confd (rw,path="default_user.conf")
/tmp/erlang-cookie-secret/ from erlang-cookie-secret (rw)
/tmp/rabbitmq-plugins/ from plugins-conf (rw)
/var/lib/rabbitmq/ from rabbitmq-erlang-cookie (rw)
/var/lib/rabbitmq/mnesia/ from persistence (rw)
/var/run/secrets/kubernetes.io/serviceaccount from hello-world-server-token-xvxcp (ro)
Containers:
rabbitmq:
Image: rabbitmq:3.8.21-management
Ports: 4369/TCP, 5672/TCP, 15672/TCP, 15692/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP
Limits:
cpu: 2
memory: 2Gi
Requests:
cpu: 1
memory: 2Gi
Readiness: tcp-socket :amqp delay=10s timeout=5s period=10s #success=1 #failure=3
Environment:
MY_POD_NAME: hello-world-server-0 (v1:metadata.name)
MY_POD_NAMESPACE: default (v1:metadata.namespace)
K8S_SERVICE_NAME: hello-world-nodes
RABBITMQ_ENABLED_PLUGINS_FILE: /operator/enabled_plugins
RABBITMQ_USE_LONGNAME: true
RABBITMQ_NODENAME: rabbit@$(MY_POD_NAME).$(K8S_SERVICE_NAME).$(MY_POD_NAMESPACE)
K8S_HOSTNAME_SUFFIX: .$(K8S_SERVICE_NAME).$(MY_POD_NAMESPACE)
Mounts:
/etc/pod-info/ from pod-info (rw)
/etc/rabbitmq/conf.d/10-operatorDefaults.conf from rabbitmq-confd (rw,path="operatorDefaults.conf")
/etc/rabbitmq/conf.d/11-default_user.conf from rabbitmq-confd (rw,path="default_user.conf")
/etc/rabbitmq/conf.d/90-userDefinedConfiguration.conf from rabbitmq-confd (rw,path="userDefinedConfiguration.conf")
/operator from rabbitmq-plugins (rw)
/var/lib/rabbitmq/ from rabbitmq-erlang-cookie (rw)
/var/lib/rabbitmq/mnesia/ from persistence (rw)
/var/run/secrets/kubernetes.io/serviceaccount from hello-world-server-token-xvxcp (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
persistence:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: persistence-hello-world-server-0
ReadOnly: false
plugins-conf:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: hello-world-plugins-conf
Optional: false
rabbitmq-confd:
Type: Projected (a volume that contains injected data from multiple sources)
ConfigMapName: hello-world-server-conf
ConfigMapOptional: <nil>
SecretName: hello-world-default-user
SecretOptionalName: <nil>
rabbitmq-erlang-cookie:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
erlang-cookie-secret:
Type: Secret (a volume populated by a Secret)
SecretName: hello-world-erlang-cookie
Optional: false
rabbitmq-plugins:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
pod-info:
Type: DownwardAPI (a volume populated by information about the pod)
Items:
metadata.labels['skipPreStopChecks'] -> skipPreStopChecks
hello-world-server-token-xvxcp:
Type: Secret (a volume populated by a Secret)
SecretName: hello-world-server-token-xvxcp
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 8m21s default-scheduler 0/4 nodes are available: 4 pod has unbound immediate PersistentVolumeClaims.
Warning FailedScheduling 8m21s default-scheduler 0/4 nodes are available: 4 pod has unbound immediate PersistentVolumeClaims.
# 手动创建PVC# persistence-hello-world-server-0# persistence-hello-world-server-0.yamlapiVersion: v1
kind: PersistentVolumeClaim
metadata:name: persistence-hello-world-server-0namespace: default
spec:accessModes:- ReadWriteOnce
storageClassName: rook-ceph-block
resources:requests:storage: 12Gi
[root@k8s-master rabbitmq]# kubectl apply -f persistence-hello-world-server-0.yaml
persistentvolumeclaim/persistence-hello-world-server-0 created
[root@k8s-master rabbitmq]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistence-hello-world-server-0 Bound pvc-91f82092-4b29-42ab-87b4-2cf3f9099089 12Gi RWO rook-ceph-block 3s
# 服务正常[root@k8s-master rabbitmq]# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/hello-world-server-0 1/1 Running 0 20m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/hello-world ClusterIP 10.100.123.79 <none>15672/TCP,15692/TCP,5672/TCP 20m
service/hello-world-nodes ClusterIP None <none>4369/TCP,25672/TCP 20m
NAME READY AGE
statefulset.apps/hello-world-server 1/1 20m
NAME ALLREPLICASREADY RECONCILESUCCESS AGE
rabbitmqcluster.rabbitmq.com/hello-world True True 20m
[root@k8s-master rabbitmq]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistence-hello-world-server-0 Bound pvc-91f82092-4b29-42ab-87b4-2cf3f9099089 12Gi RWO rook-ceph-block 2m27s
1.2.获取用户名和密码
获取用户名和密码
# 获取获取用户名和密码username="$(kubectl get secret hello-world-default-user -ojsonpath='{.data.username}'| base64 --decode)"echo"username: $username"password="$(kubectl get secret hello-world-default-user -ojsonpath='{.data.password}'| base64 --decode)"echo"password: $password"[root@k8s-master rabbitmq]# username="$(kubectl get secret hello-world-default-user -o jsonpath='{.data.username}' | base64 --decode)"[root@k8s-master rabbitmq]# echo "username: $username"
username: default_user_HGTmPURsI5VhaVjmeEQ
[root@k8s-master rabbitmq]# password="$(kubectl get secret hello-world-default-user -o jsonpath='{.data.password}' | base64 --decode)"[root@k8s-master rabbitmq]# echo "password: $password"
password: TawohNwWLpQ0D7xmpElx9bUsL_PuIhhX
# 登录信息:
username: default_user_HGTmPURsI5VhaVjmeEQ
password: TawohNwWLpQ0D7xmpElx9bUsL_PuIhhX
kubectl port-forward "service/hello-world"15672
# 系统创建service# hello-world[root@k8s-master rabbitmq]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-world ClusterIP 10.100.123.79 <none>15672/TCP,15692/TCP,5672/TCP 33m
hello-world-nodes ClusterIP None <none>4369/TCP,25672/TCP 33m
[root@k8s-master rabbitmq]# kubectl port-forward "service/hello-world" 15672
Forwarding from 127.0.0.1:15672 ->15672
Forwarding from [::1]:15672 ->15672curl -u$username:$password localhost:15672/api/overview
{"management_version":"3.8.9","rates_mode":"basic", ...}curl-udefault_user_HGTmPURsI5VhaVjmeEQ:TawohNwWLpQ0D7xmpElx9bUsL_PuIhhX10.100.123.79:15672/api/overview
# 访问[root@k8s-master rabbitmq]# curl -udefault_user_HGTmPURsI5VhaVjmeEQ:TawohNwWLpQ0D7xmpElx9bUsL_PuIhhX 10.100.123.79:15672/api/overview{"management_version":"3.8.21","rates_mode":"basic","sample_retention_policies":{"global":[600,3600,28800,86400],"basic":[600,3600],"detailed":[600]},"exchange_types":[{"name":"direct","description":"AMQP direct exchange, as per the AMQP specification","enabled":true},{"name":"fanout","description":"AMQP fanout exchange, as per the AMQP specification","enabled":true},{"name":"headers","description":"AMQP headers exchange, as per the AMQP specification","enabled":true},{"name":"topic","description":"AMQP topic exchange, as per the AMQP specification","enabled":true}],"product_version":"3.8.21","product_name":"RabbitMQ","rabbitmq_version":"3.8.21","cluster_name":"hello-world","erlang_version":"24.0.5","erlang_full_version":"Erlang/OTP 24 [erts-12.0.3] [source] [64-bit] [smp:4:2] [ds:4:2:10] [async-threads:1] [jit]","disable_stats":false,"enable_queue_totals":false,"message_stats":{},"churn_rates":{"channel_closed":0,"channel_closed_details":{"rate":0.0},"channel_created":0,"channel_created_details":{"rate":0.0},"connection_closed":312,"connection_closed_details":{"rate":0.2},"connection_created":0,"connection_created_details":{"rate":0.0},"queue_created":0,"queue_created_details":{"rate":0.0},"queue_declared":0,"queue_declared_details":{"rate":0.0},"queue_deleted":0,"queue_deleted_details":{"rate":0.0}},"queue_totals":{},"object_totals":{"channels":0,"connections":0,"consumers":0,"exchanges":7,"queues":0},"statistics_db_event_queue":0,"node":"[email protected]","listeners":[{"node":"[email protected]","protocol":"amqp","ip_address":"::","port":5672,"socket_opts":{"backlog":128,"nodelay":true,"linger":[true,0],"exit_on_close":false}},{"node":"[email protected]","protocol":"clustering","ip_address":"::","port":25672,"socket_opts":[]},{"node":"[email protected]","protocol":"http","ip_address":"::","port":15672,"socket_opts":{"cowboy_opts":{"sendfile":false},"port":15672}},{"node":"[email protected]","protocol":"http/prometheus","ip_address":"::","port":15692,"socket_opts":{"cowboy_opts":{"sendfile":false},"port":15692,"protocol":"http/prometheus"}}],"contexts":[{"ssl_opts":[],"node":"[email protected]","description":"RabbitMQ Management","path":"/","cowboy_opts":"[{sendfile,false}]","port":"15672"},{"ssl_opts":[],"node":"[email protected]","description":"RabbitMQ Prometheus","path":"/","cowboy_opts":"[{sendfile,false}]","port":"15692","protocol":"'http/prometheus'"}]}
1.3.访问RabbitMQ管理UI
rabbitmq-k8s-svc.yaml
# rabbitmq-k8s-svc.yamlapiVersion: v1
kind: Service
metadata:namespace: default
name: rabbitmq-k8s-svc
labels:statefulset.kubernetes.io/pod-name: hello-world-server-0spec:type: NodePort
ports:-port:15672name: rabbitmq-http
targetPort:15672nodePort:30672-port:5672name: rabbitmq-tcp
targetPort:5672nodePort:30673selector:statefulset.kubernetes.io/pod-name: hello-world-server-0
[root@k8s-master rabbitmq]# kubectl apply -f rabbitmq-svc.yaml
service/rabbitmq-svc created
[root@k8s-master rabbitmq]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-world ClusterIP 10.100.123.79 <none>15672/TCP,15692/TCP,5672/TCP 141m
hello-world-nodes ClusterIP None <none>4369/TCP,25672/TCP 141m
rabbitmq-k8s-svc NodePort 10.96.247.212 <none>15672:30672/TCP 6s
curl-udefault_user_HGTmPURsI5VhaVjmeEQ:TawohNwWLpQ0D7xmpElx9bUsL_PuIhhX10.96.247.212:15672/api/overview
# 地址
http://172.51.216.81:30672
username: default_user_HGTmPURsI5VhaVjmeEQ
password: TawohNwWLpQ0D7xmpElx9bUsL_PuIhhX
1.4.删除Rabbitmq
# 删除Rabbitmq# 根据创建选择
kubectl delete -f https://raw.githubusercontent.com/rabbitmq/cluster-operator/main/docs/examples/hello-world/rabbitmq.yaml
# 删除operator
kubectl delete -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml
# 查看[root@k8s-master rabbitmq]# kubectl get all -n rabbitmq-system
NAME READY STATUS RESTARTS AGE
pod/rabbitmq-cluster-operator-7cbf865f89-5c4wk 1/1 Running 1 19h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/rabbitmq-cluster-operator 1/1 11 19h
NAME DESIRED CURRENT READY AGE
replicaset.apps/rabbitmq-cluster-operator-7cbf865f89 111 19h
[root@k8s-master rabbitmq]# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/hello-world-server-0 1/1 Running 1 19h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/hello-world ClusterIP 10.100.123.79 <none>15672/TCP,15692/TCP,5672/TCP 19h
service/hello-world-nodes ClusterIP None <none>4369/TCP,25672/TCP 19h
service/rabbitmq-k8s-svc NodePort 10.106.168.227 <none>15672:30672/TCP,5672:30673/TCP 29m
NAME READY AGE
statefulset.apps/hello-world-server 1/1 19h
NAME ALLREPLICASREADY RECONCILESUCCESS AGE
rabbitmqcluster.rabbitmq.com/hello-world True True 19h
# 删除# 删除svc[root@k8s-master rabbitmq]# kubectl delete -f rabbitmq-k8s-svc.yaml service"rabbitmq-k8s-svc" deleted
# 删除rabbitmq[root@k8s-master rabbitmq]# kubectl delete -f https://raw.githubusercontent.com/rabbitmq/cluster-operator/main/docs/examples/hello-world/rabbitmq.yaml
rabbitmqcluster.rabbitmq.com "hello-world" deleted
[root@k8s-master rabbitmq]# kubectl get all
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none>443/TCP 105d
# 删除operator[root@k8s-master rabbitmq]# kubectl delete -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml
namespace "rabbitmq-system" deleted
customresourcedefinition.apiextensions.k8s.io "rabbitmqclusters.rabbitmq.com" deleted
serviceaccount "rabbitmq-cluster-operator" deleted
role.rbac.authorization.k8s.io "rabbitmq-cluster-leader-election-role" deleted
clusterrole.rbac.authorization.k8s.io "rabbitmq-cluster-operator-role" deleted
rolebinding.rbac.authorization.k8s.io "rabbitmq-cluster-leader-election-rolebinding" deleted
clusterrolebinding.rbac.authorization.k8s.io "rabbitmq-cluster-operator-rolebinding" deleted
deployment.apps "rabbitmq-cluster-operator" deleted
[root@k8s-master rabbitmq]# kubectl get all -n rabbitmq-system
No resources found in rabbitmq-system namespace.
# 删除PVC[root@k8s-master rabbitmq]# kubectl delete -f persistence-hello-world-server-0.yaml
persistentvolumeclaim "persistence-hello-world-server-0" deleted
[root@k8s-master rabbitmq]# kubectl get pvc
No resources found in default namespace.
1.5.安装方式选择
单实例安装方式
yaml文件下载地址:https://github.com/rabbitmq/cluster-operator/tree/main/docs/examples/hello-world
yaml文件名:rabbitmq.yaml
这是最简单的RabbitmqCluster定义。唯一显式指定的属性是集群的名称。其他一切都将根据集群运营商的默认值进行配置。
examples文件夹还有许多其他引用,比如用TLS、mTLS创建RabbitMQ集群,用生产默认值设置集群,添加社区插件等等。
下载地址:https://github.com/rabbitmq/cluster-operator/tree/main/docs/examples/
多种安装方式
https://github.com/rabbitmq/cluster-operator/tree/main/docs/examples
[root@k8s-master examples]# pwd
/k8s/middleware/rabbitmq/cluster-operator/docs/examples
[root@k8s-master examples]# ll
total 4
drwxr-xr-x 2 root root 59 Nov 3011:52 additionalPorts
drwxr-xr-x 2 root root 59 Nov 3011:52 community-plugins
drwxr-xr-x 2 root root 60 Nov 3011:52 custom-configuration
drwxr-xr-x 2 root root 59 Nov 3011:52 default-security-context
drwxr-xr-x 2 root root 177 Nov 3011:52 federation-over-tls
drwxr-xr-x 2 root root 60 Nov 3011:52 hello-world
drwxr-xr-x 2 root root 99 Nov 3011:52 import-definitions
drwxr-xr-x 2 root root 59 Nov 3011:52 json-log
drwxr-xr-x 2 root root 60 Nov 3011:52 mtls
drwxr-xr-x 2 root root 161 Nov 3011:52 mtls-inter-node
drwxr-xr-x 2 root root 59 Nov 3011:52 multiple-disks
drwxr-xr-x 2 root root 167 Nov 3011:52 network-policies
drwxr-xr-x 2 root root 60 Nov 3011:52 plugins
drwxr-xr-x 2 root root 60 Nov 3011:52 pod-anti-affinity
drwxr-xr-x 2 root root 114 Nov 3011:52 production-ready
-rw-r--r-- 1 root root 2372 Nov 3011:52 README.md
drwxr-xr-x 2 root root 60 Nov 3011:52 resource-limits
drwxr-xr-x 2 root root 59 Nov 3011:52 set-login-password-username
drwxr-xr-x 2 root root 59 Nov 3011:52 sidecar
drwxr-xr-x 2 root root 99 Nov 3011:52 tls
drwxr-xr-x 2 root root 60 Nov 3011:52 tolerations
drwxr-xr-x 2 root root 75 Nov 3011:52 vault-default-user
drwxr-xr-x 2 root root 75 Nov 3011:52 vault-tls
# 单实例部署
drwxr-xr-x 2 root root 60 Nov 3011:52 hello-world
# 集群部署
drwxr-xr-x 2 root root 114 Nov 3011:52 production-ready
三、实践
1.部署说明
集群部署:
https://github.com/rabbitmq/cluster-operator/tree/main/docs/examples
https://github.com/rabbitmq/cluster-operator/tree/main/docs/examples/production-ready
安装说明
https://github.com/rabbitmq/cluster-operator/blob/main/docs/examples/production-ready/README.md
[root@k8s-master examples]# cd production-ready/[root@k8s-master production-ready]# ll
total 16
-rw-r--r-- 1 root root 199 Nov 3011:52 pod-disruption-budget.yaml
-rw-r--r-- 1 root root 1184 Nov 3011:52 rabbitmq.yaml
-rw-r--r-- 1 root root 3134 Nov 3011:52 README.md
-rw-r--r-- 1 root root 409 Nov 3011:52 ssd-gke.yaml
# 部署
kubectl apply -f rabbitmq.yaml
kubectl apply -f pod-disruption-budget.yaml
配置文件
# rabbitmq.yaml[root@k8s-master production-ready]# vim rabbitmq.yaml apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:name: production-ready
spec:replicas:3resources:requests:cpu:4memory: 10Gi
limits:cpu:4memory: 10Gi
rabbitmq:additionalConfig:|
cluster_partition_handling = pause_minority
vm_memory_high_watermark_paging_ratio = 0.99
disk_free_limit.relative = 1.0
collect_statistics_interval = 10000persistence:storageClassName: ssd
storage:"500Gi"affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:-labelSelector:matchExpressions:-key: app.kubernetes.io/name
operator: In
values:- production-ready
topologyKey: kubernetes.io/hostname
override:statefulSet:spec:template:spec:containers:[]topologySpreadConstraints:-maxSkew:1topologyKey:"topology.kubernetes.io/zone"whenUnsatisfiable: DoNotSchedule
labelSelector:matchLabels:app.kubernetes.io/name: production-ready
# pod-disruption-budget.yaml [root@k8s-master production-ready]# vim pod-disruption-budget.yaml apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:name: production-ready-rabbitmq
spec:maxUnavailable:1selector:matchLabels:app.kubernetes.io/name: production-ready
2.部署RabbitMQ集群
1.下载rabbitmq/cluster-operator
# git git clone https://github.com/rabbitmq/cluster-operator.git[root@k8s-master rabbitmq]# git clone https://github.com/rabbitmq/cluster-operator.git
Cloning into 'cluster-operator'...
remote: Enumerating objects: 11892, done.
remote: Counting objects: 100% (2279/2279), done.
remote: Compressing objects: 100% (976/976), done.
remote: Total 11892(delta 1438), reused 1976(delta 1240), pack-reused 9613
Receiving objects: 100% (11892/11892), 11.95 MiB |1.66 MiB/s, done.
Resolving deltas: 100% (7740/7740), done.
2.创建Operator
# kubectl apply -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml# 创建[root@k8s-master rabbitmq]# kubectl apply -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml
namespace/rabbitmq-system created
customresourcedefinition.apiextensions.k8s.io/rabbitmqclusters.rabbitmq.com created
serviceaccount/rabbitmq-cluster-operator created
role.rbac.authorization.k8s.io/rabbitmq-cluster-leader-election-role created
clusterrole.rbac.authorization.k8s.io/rabbitmq-cluster-operator-role created
rolebinding.rbac.authorization.k8s.io/rabbitmq-cluster-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/rabbitmq-cluster-operator-rolebinding created
deployment.apps/rabbitmq-cluster-operator created
[root@k8s-master rabbitmq]# kubectl get all -n rabbitmq-system
NAME READY STATUS RESTARTS AGE
pod/rabbitmq-cluster-operator-7cbf865f89-s95gq 1/1 Running 0 33s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/rabbitmq-cluster-operator 1/1 11 33s
NAME DESIRED CURRENT READY AGE
replicaset.apps/rabbitmq-cluster-operator-7cbf865f89 111 33s
3.修改配置
# rabbitmq.yaml# cpu、memory# storageClassName: rook-ceph-block[root@k8s-master production-ready]# vim rabbitmq.yaml apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:name: production-ready
spec:replicas:3resources:requests:cpu:1memory: 2Gi
limits:cpu:1memory: 2Gi
rabbitmq:additionalConfig:|
cluster_partition_handling = pause_minority
vm_memory_high_watermark_paging_ratio = 0.99
disk_free_limit.relative = 1.0
collect_statistics_interval = 10000persistence:storageClassName: rook-ceph-block
storage:"12Gi"
4.使用cluster-operator创建RabbitMQ集群
[root@k8s-master production-ready]# pwd
/k8s/middleware/rabbitmq/cluster-operator/docs/examples/production-ready
[root@k8s-master production-ready]# ll
total 16
-rw-r--r-- 1 root root 199 Nov 3011:52 pod-disruption-budget.yaml
-rw-r--r-- 1 root root 1193 Dec 110:28 rabbitmq.yaml
-rw-r--r-- 1 root root 3134 Nov 3011:52 README.md
-rw-r--r-- 1 root root 409 Nov 3011:52 ssd-gke.yaml
# 创建[root@k8s-master production-ready]# kubectl apply -f rabbitmq.yaml
rabbitmqcluster.rabbitmq.com/production-ready created
[root@k8s-master production-ready]# [root@k8s-master production-ready]# kubectl apply -f pod-disruption-budget.yaml
poddisruptionbudget.policy/production-ready-rabbitmq created
# 查看[root@k8s-master production-ready]# kubectl get all -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/production-ready-server-0 1/1 Running 0 50s 10.244.36.117 k8s-node1 <none><none>
pod/production-ready-server-1 1/1 Running 0 50s 10.244.169.164 k8s-node2 <none><none>
pod/production-ready-server-2 1/1 Running 0 50s 10.244.107.251 k8s-node3 <none><none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none>443/TCP 105d <none>
service/production-ready ClusterIP 10.111.187.65 <none>5672/TCP,15672/TCP,15692/TCP 50s app.kubernetes.io/name=production-ready
service/production-ready-nodes ClusterIP None <none>4369/TCP,25672/TCP 50s app.kubernetes.io/name=production-ready
NAME READY AGE CONTAINERS IMAGES
statefulset.apps/production-ready-server 3/3 50s rabbitmq rabbitmq:3.8.21-management
NAME ALLREPLICASREADY RECONCILESUCCESS AGE
rabbitmqcluster.rabbitmq.com/production-ready True True 50s
[root@k8s-master production-ready]# kubectl get all -n rabbitmq-system
NAME READY STATUS RESTARTS AGE
pod/rabbitmq-cluster-operator-7cbf865f89-s95gq 1/1 Running 0 105m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/rabbitmq-cluster-operator 1/1 11 105m
NAME DESIRED CURRENT READY AGE
replicaset.apps/rabbitmq-cluster-operator-7cbf865f89 111 105m
# PVC[root@k8s-master production-ready]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistence-production-ready-server-0 Bound pvc-71d4c7b8-554c-4f9e-856e-325e80ed88f2 12Gi RWO rook-ceph-block 6m10s
persistence-production-ready-server-1 Bound pvc-6be2c9b3-a0b0-43ae-af1f-74ae92bce5ce 12Gi RWO rook-ceph-block 6m10s
persistence-production-ready-server-2 Bound pvc-2243a65e-f38e-4164-a2f1-ec0a65be486d 12Gi RWO rook-ceph-block 6m10s
3.获取用户名和密码
获取用户名和密码
# user
kubectl -n NAMESPACE get secret INSTANCE-default-user -ojsonpath="{.data.username}"| base64 --decode# pass
kubectl -n NAMESPACE get secret INSTANCE-default-user -ojsonpath="{.data.password}"| base64 --decode# 获取获取用户名和密码 production-readyusername="$(kubectl get secret production-ready-default-user -ojsonpath='{.data.username}'| base64 --decode)"echo"username: $username"password="$(kubectl get secret production-ready-default-user -ojsonpath='{.data.password}'| base64 --decode)"echo"password: $password"[root@k8s-master production-ready]# username="$(kubectl get secret production-ready-default-user -o jsonpath='{.data.username}' | base64 --decode)"[root@k8s-master production-ready]# echo "username: $username"
username: default_user_vmOg0aiF8VarwVhb0sE
[root@k8s-master production-ready]# password="$(kubectl get secret production-ready-default-user -o jsonpath='{.data.password}' | base64 --decode)"[root@k8s-master production-ready]# echo "password: $password"
password: rKr9N2v-Cxrec78zo1LUnigjyOla_YLv
# 登录信息:
username: default_user_vmOg0aiF8VarwVhb0sE
password: rKr9N2v-Cxrec78zo1LUnigjyOla_YLv
kubectl port-forward "service/production-ready"15672
# 系统创建service# production-ready[root@k8s-master production-ready]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
production-ready ClusterIP 10.111.187.65 <none>5672/TCP,15672/TCP,15692/TCP 6m48s
production-ready-nodes ClusterIP None <none>4369/TCP,25672/TCP 6m48s
[root@k8s-master production-ready]# kubectl port-forward "service/production-ready" 15672
Forwarding from 127.0.0.1:15672 ->15672
Forwarding from [::1]:15672 ->15672curl -u$username:$password localhost:15672/api/overview
{"management_version":"3.8.9","rates_mode":"basic", ...}curl -udefault_user_vmOg0aiF8VarwVhb0sE:rKr9N2v-Cxrec78zo1LUnigjyOla_YLv 10.111.187.65:15672/api/overview
# 访问[root@k8s-master production-ready]# curl -udefault_user_vmOg0aiF8VarwVhb0sE:rKr9N2v-Cxrec78zo1LUnigjyOla_YLv 10.111.187.65:15672/api/overview{"management_version":"3.8.21","rates_mode":"basic","sample_retention_policies":{"global":[600,3600,28800,86400],"basic":[600,3600],"detailed":[600]},"exchange_types":[{"name":"direct","description":"AMQP direct exchange, as per the AMQP specification","enabled":true},{"name":"fanout","description":"AMQP fanout exchange, as per the AMQP specification","enabled":true},{"name":"headers","description":"AMQP headers exchange, as per the AMQP specification","enabled":true},{"name":"topic","description":"AMQP topic exchange, as per the AMQP specification","enabled":true}],"product_version":"3.8.21","product_name":"RabbitMQ","rabbitmq_version":"3.8.21","cluster_name":"production-ready","erlang_version":"24.0.5","erlang_full_version":"Erlang/OTP 24 [erts-12.0.3] [source] [64-bit] [smp:4:1] [ds:4:1:10] [async-threads:1] [jit]","disable_stats":false,"enable_queue_totals":false,"message_stats":{},"churn_rates":{"channel_closed":0,"channel_closed_details":{"rate":0.0},"channel_created":0,"channel_created_details":{"rate":0.0},"connection_closed":153,"connection_closed_details":{"rate":0.2},"connection_created":0,"connection_created_details":{"rate":0.0},"queue_created":0,"queue_created_details":{"rate":0.0},"queue_declared":0,"queue_declared_details":{"rate":0.0},"queue_deleted":0,"queue_deleted_details":{"rate":0.0}},"queue_totals":{},"object_totals":{"channels":0,"connections":0,"consumers":0,"exchanges":7,"queues":0},"statistics_db_event_queue":0,"node":"[email protected]","listeners":[{"node":"[email protected]","protocol":"amqp","ip_address":"::","port":5672,"socket_opts":{"backlog":128,"nodelay":true,"linger":[true,0],"exit_on_close":false}},{"node":"[email protected]","protocol":"amqp","ip_address":"::","port":5672,"socket_opts":{"backlog":128,"nodelay":true,"linger":[true,0],"exit_on_close":false}},{"node":"[email protected]","protocol":"amqp","ip_address":"::","port":5672,"socket_opts":{"backlog":128,"nodelay":true,"linger":[true,0],"exit_on_close":false}},{"node":"[email protected]","protocol":"clustering","ip_address":"::","port":25672,"socket_opts":[]},{"node":"[email protected]","protocol":"clustering","ip_address":"::","port":25672,"socket_opts":[]},{"node":"[email protected]","protocol":"clustering","ip_address":"::","port":25672,"socket_opts":[]},{"node":"[email protected]","protocol":"http","ip_address":"::","port":15672,"socket_opts":{"cowboy_opts":{"sendfile":false},"port":15672}},{"node":"[email protected]","protocol":"http","ip_address":"::","port":15672,"socket_opts":{"cowboy_opts":{"sendfile":false},"port":15672}},{"node":"[email protected]","protocol":"http","ip_address":"::","port":15672,"socket_opts":{"cowboy_opts":{"sendfile":false},"port":15672}},{"node":"[email protected]","protocol":"http/prometheus","ip_address":"::","port":15692,"socket_opts":{"cowboy_opts":{"sendfile":false},"port":15692,"protocol":"http/prometheus"}},{"node":"[email protected]","protocol":"http/prometheus","ip_address":"::","port":15692,"socket_opts":{"cowboy_opts":{"sendfile":false},"port":15692,"protocol":"http/prometheus"}},{"node":"[email protected]","protocol":"http/prometheus","ip_address":"::","port":15692,"socket_opts":{"cowboy_opts":{"sendfile":false},"port":15692,"protocol":"http/prometheus"}}],"contexts":[{"ssl_opts":[],"node":"[email protected]","description":"RabbitMQ Management","path":"/","cowboy_opts":"[{sendfile,false}]","port":"15672"},{"ssl_opts":[],"node":"[email protected]","description":"RabbitMQ Management","path":"/","cowboy_opts":"[{sendfile,false}]","port":"15672"},{"ssl_opts":[],"node":"[email protected]","description":"RabbitMQ Management","path":"/","cowboy_opts":"[{sendfile,false}]","port":"15672"},{"ssl_opts":[],"node":"[email protected]","description":"RabbitMQ Prometheus","path":"/","cowboy_opts":"[{sendfile,false}]","port":"15692","protocol":"'http/prometheus'"},{"ssl_opts":[],"node":"[email protected]","description":"RabbitMQ Prometheus","path":"/","cowboy_opts":"[{sendfile,false}]","port":"15692","protocol":"'http/prometheus'"},{"ssl_opts":[],"node":"[email protected]","description":"RabbitMQ Prometheus","path":"/","cowboy_opts":"[{sendfile,false}]","port":"15692","protocol":"'http/prometheus'"}]}[root@k8s-master production-ready]#
4.访问RabbitMQ管理UI
rabbitmq-cluster-svc.yaml
# rabbitmq-cluster-svc.yamlapiVersion: v1
kind: Service
metadata:namespace: default
name: rabbitmq-cluster-svc
labels:app.kubernetes.io/name: production-ready
spec:type: NodePort
ports:-port:15672name: rabbitmq-http
targetPort:15672nodePort:30672-port:5672name: rabbitmq-tcp
targetPort:5672nodePort:30673selector:app.kubernetes.io/name: production-ready
[root@k8s-master rabbitmq]# kubectl apply -f rabbitmq-svc.yaml
service/rabbitmq-svc created
[root@k8s-master rabbitmq]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
production-ready ClusterIP 10.111.187.65 <none>5672/TCP,15672/TCP,15692/TCP 10m
production-ready-nodes ClusterIP None <none>4369/TCP,25672/TCP 10m
rabbitmq-cluster-svc NodePort 10.106.36.141 <none>15672:30672/TCP,5672:30673/TCP 5s
curl -udefault_user_vmOg0aiF8VarwVhb0sE:rKr9N2v-Cxrec78zo1LUnigjyOla_YLv 10.106.36.141:15672/api/overview
# 地址
http://172.51.216.81:30672
username: default_user_vmOg0aiF8VarwVhb0sE
password: rKr9N2v-Cxrec78zo1LUnigjyOla_YLv
5.扩容
修改配置
# rabbitmq.yaml# cpu、memory# storageClassName: rook-ceph-block[root@k8s-master production-ready]# vim rabbitmq.yaml apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:name: production-ready
spec:replicas:4resources:requests:cpu:1memory: 2Gi
limits:cpu:1memory: 2Gi
rabbitmq:additionalConfig:|
cluster_partition_handling = pause_minority
vm_memory_high_watermark_paging_ratio = 0.99
disk_free_limit.relative = 1.0
collect_statistics_interval = 10000persistence:storageClassName: rook-ceph-block
storage:"12Gi"
[root@k8s-master production-ready]# kubectl apply -f rabbitmq.yaml
rabbitmqcluster.rabbitmq.com/production-ready configured
# 查看[root@k8s-master production-ready]# kubectl get all -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/production-ready-server-0 1/1 Running 0 24m 10.244.36.117 k8s-node1 <none><none>
pod/production-ready-server-1 1/1 Running 0 24m 10.244.169.164 k8s-node2 <none><none>
pod/production-ready-server-2 1/1 Running 0 24m 10.244.107.251 k8s-node3 <none><none>
pod/production-ready-server-3 1/1 Running 0 45s 10.244.169.159 k8s-node2 <none><none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none>443/TCP 105d <none>
service/production-ready ClusterIP 10.111.187.65 <none>5672/TCP,15672/TCP,15692/TCP 24m app.kubernetes.io/name=production-ready
service/production-ready-nodes ClusterIP None <none>4369/TCP,25672/TCP 24m app.kubernetes.io/name=production-ready
service/rabbitmq-cluster-svc NodePort 10.106.36.141 <none>15672:30672/TCP,5672:30673/TCP 14m app.kubernetes.io/name=production-ready
NAME READY AGE CONTAINERS IMAGES
statefulset.apps/production-ready-server 4/4 24m rabbitmq rabbitmq:3.8.21-management
NAME ALLREPLICASREADY RECONCILESUCCESS AGE
rabbitmqcluster.rabbitmq.com/production-ready True True 24m
[root@k8s-master production-ready]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistence-production-ready-server-0 Bound pvc-71d4c7b8-554c-4f9e-856e-325e80ed88f2 12Gi RWO rook-ceph-block 27m
persistence-production-ready-server-1 Bound pvc-6be2c9b3-a0b0-43ae-af1f-74ae92bce5ce 12Gi RWO rook-ceph-block 27m
persistence-production-ready-server-2 Bound pvc-2243a65e-f38e-4164-a2f1-ec0a65be486d 12Gi RWO rook-ceph-block 27m
persistence-production-ready-server-3 Bound pvc-26fca32f-d0d9-44af-8378-1a2aeefb1d94 12Gi RWO rook-ceph-block 3m21s
6.缩容
修改配置
# rabbitmq.yaml# cpu、memory# storageClassName: rook-ceph-block[root@k8s-master production-ready]# vim rabbitmq.yaml apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:name: production-ready
spec:replicas:3resources:requests:cpu:1memory: 2Gi
limits:cpu:1memory: 2Gi
rabbitmq:additionalConfig:|
cluster_partition_handling = pause_minority
vm_memory_high_watermark_paging_ratio = 0.99
disk_free_limit.relative = 1.0
collect_statistics_interval = 10000persistence:storageClassName: rook-ceph-block
storage:"12Gi"
[root@k8s-master production-ready]# kubectl apply -f rabbitmq.yaml
rabbitmqcluster.rabbitmq.com/production-ready configured
# 查看[root@k8s-master production-ready]# kubectl get all -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/production-ready-server-0 1/1 Running 0 24m 10.244.36.117 k8s-node1 <none><none>
pod/production-ready-server-1 1/1 Running 0 24m 10.244.169.164 k8s-node2 <none><none>
pod/production-ready-server-2 1/1 Running 0 24m 10.244.107.251 k8s-node3 <none><none>
pod/production-ready-server-3 1/1 Running 0 45s 10.244.169.159 k8s-node2 <none><none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none>443/TCP 105d <none>
service/production-ready ClusterIP 10.111.187.65 <none>5672/TCP,15672/TCP,15692/TCP 24m app.kubernetes.io/name=production-ready
service/production-ready-nodes ClusterIP None <none>4369/TCP,25672/TCP 24m app.kubernetes.io/name=production-ready
service/rabbitmq-cluster-svc NodePort 10.106.36.141 <none>15672:30672/TCP,5672:30673/TCP 14m app.kubernetes.io/name=production-ready
NAME READY AGE CONTAINERS IMAGES
statefulset.apps/production-ready-server 4/4 24m rabbitmq rabbitmq:3.8.21-management
NAME ALLREPLICASREADY RECONCILESUCCESS AGE
rabbitmqcluster.rabbitmq.com/production-ready True True 24m
[root@k8s-master production-ready]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistence-production-ready-server-0 Bound pvc-71d4c7b8-554c-4f9e-856e-325e80ed88f2 12Gi RWO rook-ceph-block 27m
persistence-production-ready-server-1 Bound pvc-6be2c9b3-a0b0-43ae-af1f-74ae92bce5ce 12Gi RWO rook-ceph-block 27m
persistence-production-ready-server-2 Bound pvc-2243a65e-f38e-4164-a2f1-ec0a65be486d 12Gi RWO rook-ceph-block 27m
persistence-production-ready-server-3 Bound pvc-26fca32f-d0d9-44af-8378-1a2aeefb1d94 12Gi RWO rook-ceph-block 3m21s
缩容不起作用。
7.Spring Boot测试
7.1.服务配置
msa-k8s-rabbitmq.yaml
apiVersion: v1
kind: Service
metadata:namespace: dev
name: msa-k8s-rabbitmq
labels:app: msa-k8s-rabbitmq
spec:type: ClusterIP
ports:-port:8143targetPort:8143selector:app: msa-k8s-rabbitmq
---apiVersion: apps/v1
kind: Deployment
metadata:namespace: dev
name: msa-k8s-rabbitmq
spec:replicas:3selector:matchLabels:app: msa-k8s-rabbitmq
template:metadata:labels:app: msa-k8s-rabbitmq
spec:imagePullSecrets:-name: harborsecret #对应创建私有镜像密钥Secretcontainers:-name: msa-k8s-redis
image: 172.51.216.85:8888/springcloud/msa-k8s-rabbitmq:1.0.0
imagePullPolicy: Always #如果省略imagePullPolicy,策略为IfNotPresentports:-containerPort:8143env:-name: ACTIVE
value:"-Dspring.profiles.active=dev"
application-k8s.yml
# Redis集群配置server:port:8143spring:application:name: msa-k8s-rabbitmq
#配置rabbitMq 服务器rabbitmq:host: production-ready.default.svc.cluster.local
port:5672username: default_user_IdbGSVeW1plrpskejhI
password: mYaPNH_Hlb-g1rNYMiPWGdKex-3l8po5
#虚拟host 可以不设置,使用server默认host#virtual-host: JCcccHost# 说明# 集群内部访问:
production-ready
#完整配置host: production-ready.default.svc.cluster.local
# 或host: rabbitmq-cluster-svc.default.svc.cluster.local
[root@k8s-master rabbitmq]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
production-ready ClusterIP 10.102.238.59 <none> 5672/TCP,15672/TCP,15692/TCP 89m
production-ready-nodes ClusterIP None <none> 4369/TCP,25672/TCP 89m
rabbitmq-cluster-svc NodePort 10.106.36.141 <none> 15672:30672/TCP,5672:30673/TCP 145m
# 集群外部访问,需要每个实例配置一个service,暴露端口# 参考erver:port:8143spring:application:name: msa-k8s-rabbitmq
rabbitmq:# 集群地址,用逗号分隔addresses: 192.168.11.71:5672,192.168.11.72:5672,192.168.11.71:5673connection-timeout:15000password: guest
# 使用启用消息确认模式publisher-confirms:trueusername: guest
virtual-host: /
# 配置每个实例的service addresses: 192.168.11.71:5672,192.168.11.72:5672,192.168.11.71:5673
7.2.创建服务
docker build -t172.51.216.85:8888/springcloud/msa-k8s-rabbitmq:1.1.0 .docker push 172.51.216.85:8888/springcloud/msa-k8s-rabbitmq:1.1.0
[root@k8s-master operator]# kubectl apply -f msa-k8s-rabbitmq.yaml
service/msa-k8s-redis created
deployment.apps/msa-k8s-redis created
root@k8s-master rabbitmq]# kubectl get all -n dev | grep msa-k8s-rabbitmq
pod/msa-k8s-rabbitmq-84976bbd47-8mh6p 1/1 Running 0 8m25s
pod/msa-k8s-rabbitmq-84976bbd47-9swjn 1/1 Running 0 8m25s
pod/msa-k8s-rabbitmq-84976bbd47-nsqbt 1/1 Running 0 8m25s
service/msa-k8s-rabbitmq ClusterIP 10.107.32.2 <none>8143/TCP 8m25s
deployment.apps/msa-k8s-rabbitmq 3/3 33 8m25s
replicaset.apps/msa-k8s-rabbitmq-84976bbd47 333 8m25s
# 测试curl10.107.32.2:8143/send
[root@k8s-master rabbitmq]# curl 10.107.32.2:8143/send
发送消息成功!
[root@k8s-master rabbitmq]# curl 10.107.32.2:8143/send
发送消息成功!
[root@k8s-master rabbitmq]# kubectl logs -f msa-k8s-rabbitmq-84976bbd47-8mh6p
Error from server (NotFound): pods "msa-k8s-rabbitmq-84976bbd47-8mh6p" not found
[root@k8s-master rabbitmq]# kubectl logs -f msa-k8s-rabbitmq-84976bbd47-8mh6p -n dev
OpenJDK 64-Bit Server VM warning: ignoring option PermSize=512M; support was removed in8.0
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in8.0. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ |'_| | '_ \/ _` |\\\\\\/ ___)||_)|||||||(_||))))' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.3.10.RELEASE)
2021-12-01 14:15:22.184 INFO 1 --- [ main] com.k8s.msa.MsaK8sRabbitmqApplication : Starting MsaK8sRabbitmqApplication v1.0.0 on msa-k8s-rabbitmq-84976bbd47-8mh6p with PID 1 (/app.jar started by root in /)
2021-12-01 14:15:22.189 INFO 1 --- [ main] com.k8s.msa.MsaK8sRabbitmqApplication : The following profiles are active: dev
2021-12-01 14:15:23.880 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8143 (http)
2021-12-01 14:15:23.896 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2021-12-01 14:15:23.896 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.45]
2021-12-01 14:15:23.999 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2021-12-01 14:15:23.999 INFO 1 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1721 ms
2021-12-01 14:15:24.898 INFO 1 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'
2021-12-01 14:15:25.349 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8143 (http) with context path ''
2021-12-01 14:15:25.351 INFO 1 --- [ main] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [production-ready.default.svc.cluster.local:5672]2021-12-01 14:15:25.447 INFO 1 --- [ main] o.s.a.r.c.CachingConnectionFactory : Created new connection: rabbitConnectionFactory#185a6e9:0/SimpleConnection@6cd28fa7 [delegate=amqp://[email protected]:5672/, localPort= 33022]2021-12-01 14:15:25.530 INFO 1 --- [ main] com.k8s.msa.MsaK8sRabbitmqApplication : Started MsaK8sRabbitmqApplication in4.14 seconds (JVM running for4.824)2021-12-01 14:16:30.850 INFO 1 --- [nio-8143-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]: Initializing Spring DispatcherServlet 'dispatcherServlet'2021-12-01 14:16:30.850 INFO 1 --- [nio-8143-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'2021-12-01 14:16:30.856 INFO 1 --- [nio-8143-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in6 ms
发送消息成功!:send,this is first messge! $$$ -----V2Wed Dec 01 14:16:30 CST 2021
发送消息成功!:send,this is first messge! $$$ -----V2Wed Dec 01 14:16:32 CST 2021
消费者消费消息了:send,this is first messge! $$$ -----V2Wed Dec 01 14:16:32 CST 2021
发送消息成功!:send,this is first messge! $$$ -----V2Wed Dec 01 14:16:35 CST 2021
消费者消费消息了:send,this is first messge! $$$ -----V2Wed Dec 01 14:16:35 CST 2021
发送消息成功!:send,this is first messge! $$$ -----V2Wed Dec 01 14:16:38 CST 2021
消费者消费消息了:send,this is first messge! $$$ -----V2Wed Dec 01 14:16:40 CST 2021
发送消息成功!:send,this is first messge! $$$ -----V2Wed Dec 01 14:16:41 CST 2021
消费者消费消息了:send,this is first messge! $$$ -----V2Wed Dec 01 14:16:44 CST 2021
7.3.总结
1.开发测试:
每个RabbitMQ实例单独配置service,以NodePort方式暴露端口
2.集群内服访问
# Spring Boot# 集群内部访问:
production-ready
#完整配置
host: production-ready.default.svc.cluster.local
# 或
host: rabbitmq-cluster-svc.default.svc.cluster.local
8.删除集群
# 删除Rabbitmq[root@k8s-master production-ready]# pwd
/k8s/middleware/rabbitmq/cluster-operator/docs/examples/production-ready
[root@k8s-master production-ready]# ll
total 16
-rw-r--r-- 1 root root 199 Nov 3011:52 pod-disruption-budget.yaml
-rw-r--r-- 1 root root 500 Dec 112:14 rabbitmq.yaml
-rw-r--r-- 1 root root 3134 Nov 3011:52 README.md
-rw-r--r-- 1 root root 409 Nov 3011:52 ssd-gke.yaml
# 删除
kubectl delete -f pod-disruption-budget.yaml
kubectl delete -f rabbitmq.yaml
# 删除operator
kubectl delete -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml
# 查看[root@k8s-master production-ready]# kubectl get all -n rabbitmq-system
NAME READY STATUS RESTARTS AGE
pod/rabbitmq-cluster-operator-7cbf865f89-s95gq 1/1 Running 0 4h31m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/rabbitmq-cluster-operator 1/1 11 4h31m
NAME DESIRED CURRENT READY AGE
replicaset.apps/rabbitmq-cluster-operator-7cbf865f89 111 4h31m
[root@k8s-master production-ready]# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/production-ready-server-0 1/1 Running 0 102m
pod/production-ready-server-1 1/1 Running 0 102m
pod/production-ready-server-2 1/1 Running 0 102m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none>443/TCP 106d
service/production-ready ClusterIP 10.102.238.59 <none>5672/TCP,15672/TCP,15692/TCP 102m
service/production-ready-nodes ClusterIP None <none>4369/TCP,25672/TCP 102m
service/rabbitmq-cluster-svc NodePort 10.106.36.141 <none>15672:30672/TCP,5672:30673/TCP 158m
NAME READY AGE
statefulset.apps/production-ready-server 3/3 102m
NAME ALLREPLICASREADY RECONCILESUCCESS AGE
rabbitmqcluster.rabbitmq.com/production-ready True True 102m
[root@k8s-master production-ready]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistence-production-ready-server-0 Bound pvc-0c3c4767-669c-497e-a6a7-c6ee4b6ff283 12Gi RWO rook-ceph-block 102m
persistence-production-ready-server-1 Bound pvc-8ecc6b70-26ff-483d-a456-2325dd4376b3 12Gi RWO rook-ceph-block 102m
persistence-production-ready-server-2 Bound pvc-6a9f3349-0396-46c1-a7a9-7fd8ad9c218b 12Gi RWO rook-ceph-block 102m
# 删除# 删除svc[root@k8s-master rabbitmq]# kubectl delete -f rabbitmq-cluster-svc.yaml service"rabbitmq-cluster-svc" deleted
# 删除rabbitmq[root@k8s-master production-ready]# kubectl delete -f pod-disruption-budget.yaml
poddisruptionbudget.policy "production-ready-rabbitmq" deleted
[root@k8s-master production-ready]# kubectl delete -f rabbitmq.yaml
rabbitmqcluster.rabbitmq.com "production-ready" deleted
[root@k8s-master rabbitmq]# kubectl get all
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none>443/TCP 105d
# 删除operator[root@k8s-master rabbitmq]# kubectl delete -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml
namespace "rabbitmq-system" deleted
customresourcedefinition.apiextensions.k8s.io "rabbitmqclusters.rabbitmq.com" deleted
serviceaccount "rabbitmq-cluster-operator" deleted
role.rbac.authorization.k8s.io "rabbitmq-cluster-leader-election-role" deleted
clusterrole.rbac.authorization.k8s.io "rabbitmq-cluster-operator-role" deleted
rolebinding.rbac.authorization.k8s.io "rabbitmq-cluster-leader-election-rolebinding" deleted
clusterrolebinding.rbac.authorization.k8s.io "rabbitmq-cluster-operator-rolebinding" deleted
deployment.apps "rabbitmq-cluster-operator" deleted
[root@k8s-master rabbitmq]# kubectl get all -n rabbitmq-system
No resources found in rabbitmq-system namespace.
[root@k8s-master rabbitmq]# kubectl get pvc
No resources found in default namespace.
- Kubernetes
# Kubernetes
https://iothub.org.cn/docs/kubernetes/
https://iothub.org.cn/docs/kubernetes/middleware/rabbitmq/
版权归原作者 IoTHub - 物联网开源技术社区 所有, 如有侵权,请联系我们删除。