往期回顾:
云端技术驾驭DAY01——云计算底层技术奥秘、云服务器磁盘技术、虚拟化管理、公有云概述
云端技术驾驭DAY02——华为云管理、云主机管理、跳板机配置、制作私有镜像模板
云端技术驾驭DAY03——云主机网站部署、web集群部署、Elasticsearch安装
云端技术驾驭DAY04——Logstash安装部署及插件模块
云端技术驾驭DAY06——容器技术概述、镜像与容器管理、定制简单镜像、容器内安装部署服务
云端技术驾驭DAY07——Dockerfile详解、容器镜像制作、私有仓库
云端技术驾驭DAY08——部署容器服务、Compose微服务管理、harbor仓库部署及管理
云端技术驾驭DAY09——k8s集群安装部署、calico插件部署、计算节点配置管理
云端技术驾驭DAY10——kubectl命令详解、Pod创建过程、Pod的生命周期、定制Pod、资源对象文件
云端技术驾驭DAY11——资源对象文件、Pod自定义命令、多容器Pod、资源监控工具
云端技术驾驭DAY12——Pod调度策略、Pod标签管理、Pod资源配额与限额、全局资源配额与限额策略
云端技术驾驭DAY13
Pod调度策略管理
污点容忍策略
污点概述
- 什么是污点? - 污点(Taint)是使节点与Pod产生排斥的一类规则
- 污点策略是如何实现 - 污点策略通过嵌合在键值对上的污点标签进行声明
- 污点标签 - 尽量不调度:
PreferNoSchedule
- 不会被调度:NoSchedule
- 驱逐节点:NoExecute
- 管理污点标签 - 污点标签必须绑定在键值对上,格式为:
key=value:污点标签
- 查看污点标签:kubectl describe nodes [节点名字]
- 设置污点标签:kubectl taint node [节点名字] key=value:污点标签
- 删除污点标签:kubectl taint node [节点名字] key=value:污点标签-
污点策略
- 污点验证 - 为node-0001设置PreferNoschedule污点标签- 为node-0002设置NoSchedule污点标签
[root@master ~]# kubectl taint node node-0001 k=v1:PreferNoSchedule
node/node-0001 tainted
[root@master ~]# kubectl taint node node-0002 k=v2:NoSchedule
node/node-0002 tainted
[root@master ~]# kubectl describe nodes | grep Taints
Taints: node-role.kubernetes.io/control-plane:NoSchedule
Taints: k=v1:PreferNoSchedule
Taints: k=v2:NoSchedule
Taints: <none>
Taints: <none>
Taints: <none>
- Pod资源文件
[root@master ~]# vim myphp.yaml
---
kind: Pod
apiVersion: v1
metadata:
name: myphp
spec:
containers:
- name: php
image: myos:php-fpm
resources:
requests:
cpu: 1500m
- 验证测试
[root@master ~]# sed "s,myphp,php1," myphp.yaml | kubectl apply -f -
pod/php1 created
[root@master ~]# sed "s,myphp,php2," myphp.yaml | kubectl apply -f -
pod/php2 created
[root@master ~]# sed "s,myphp,php3," myphp.yaml | kubectl apply -f - // 创建三个Pod
pod/php3 created
[root@master ~]# kubectl get pods -o wide // 发现会优先使用没有污点的节点
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
php1 1/1 Running 0 35s 10.244.240.134 node-0004 <none> <none>
php2 1/1 Running 0 30s 10.244.243.209 node-0003 <none> <none>
php3 1/1 Running 0 26s 10.244.153.139 node-0005 <none> <none>
- 在没有其他节点可用的时候,使用PreferNoSchedule污点所在的节点运行Pod
[root@master ~]# sed "s,myphp,php4," myphp.yaml | kubectl apply -f -
pod/php4 created
[root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
php1 1/1 Running 0 4m19s 10.244.240.134 node-0004 <none> <none>
php2 1/1 Running 0 4m14s 10.244.243.209 node-0003 <none> <none>
php3 1/1 Running 0 4m10s 10.244.153.139 node-0005 <none> <none>
php4 1/1 Running 0 2s 10.244.21.132 node-0001 <none> <none>
- 继续创建Pod - 即使Pod创建失败,也不会使用NoSchedule所在节点
[root@master ~]# sed "s,myphp,php5," myphp.yaml | kubectl apply -f -
pod/php5 created
[root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
php1 1/1 Running 0 5m37s 10.244.240.134 node-0004 <none> <none>
php2 1/1 Running 0 5m32s 10.244.243.209 node-0003 <none> <none>
php3 1/1 Running 0 5m28s 10.244.153.139 node-0005 <none> <none>
php4 1/1 Running 0 80s 10.244.21.132 node-0001 <none> <none>
php5 0/1 Pending 0 2s <none> <none> <none> <none>
- NoSchedule策略 - 设置NoSchedule污点标签只对新建Pod有效,对于已经创建完成的Pod不会产生影响
[root@master ~]# kubectl taint node node-0003 k=v3:NoSchedule
node/node-0003 tainted
[root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
php1 1/1 Running 0 8m38s 10.244.240.134 node-0004 <none> <none>
php2 1/1 Running 0 8m33s 10.244.243.209 node-0003 <none> <none>
php3 1/1 Running 0 8m29s 10.244.153.139 node-0005 <none> <none>
php4 1/1 Running 0 4m21s 10.244.21.132 node-0001 <none> <none>
php5 0/1 Pending 0 3m3s <none> <none> <none> <none>
- 驱逐策略 - 驱逐策略会删除该节点上的所有Pod
[root@master ~]# kubectl taint node node-0004 k=v4:NoExecute // 为node-0004设置NoExecute策略
node/node-0004 tainted
[root@master ~]# kubectl describe nodes | grep Taints
Taints: node-role.kubernetes.io/control-plane:NoSchedule
Taints: k=v1:PreferNoSchedule
Taints: k=v2:NoSchedule
Taints: k=v3:NoSchedule
Taints: k=v4:NoExecute
Taints: <none>
[root@master ~]# kubectl get pods -o wide // 查看Pod情况,NoExecute污点所在节点的Pod已经被删除了
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
php2 1/1 Running 0 15m 10.244.243.209 node-0003 <none> <none>
php3 1/1 Running 0 15m 10.244.153.139 node-0005 <none> <none>
php4 1/1 Running 0 11m 10.244.21.132 node-0001 <none> <none>
php5 0/1 Pending 0 9m57s <none> <none> <none> <none>
- 删除实验所有污点策略
[root@master ~]# kubectl taint node node-000{1..4} k-
node/node-0001 untainted
node/node-0002 untainted
node/node-0003 untainted
node/node-0004 untainted
容忍策略
- 容忍策略是什么? - 容忍策略与污点策略相反,某些时候我们需要在有污点的节点上运行Pod,这种无视污点标签的调度方式称为容忍策略
- 容忍策略实验 - 节点node-000{1…2}设置污点标签
k=v1:NoSchedule
- 节点node-000{3…4}设置污点标签k=v2:NoSchedule
- 节点node-0005设置污点标签k=v1:NoExecute
- 精确匹配策略
[root@master ~]# kubectl taint node node-000{1..2} k=v1:NoSchedule
node/node-0001 tainted
node/node-0002 tainted
[root@master ~]# kubectl taint node node-000{3..4} k=v2:NoSchedule
node/node-0003 tainted
node/node-0004 tainted
[root@master ~]# kubectl taint node node-0005 k=v1:NoExecute
node/node-0005 tainted
[root@master ~]# vim myphp.yaml // 容忍k=v1:NoSchedule污点
---
kind: Pod
apiVersion: v1
metadata:
name: myphp
spec:
tolerations:
- operator: Equal // 完全匹配键值对
key: k // 键
value: v1 // 值
effect: NoSchedule // 污点标签
containers:
- name: php
image: myos:php-fpm
resources:
requests:
cpu: 1500m
[root@master ~]# for i in php{1..3};do sed "s,myphp,${i}," myphp.yaml ;done|kubectl apply -f -
pod/php1 created
pod/php2 created
pod/php3 created
[root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
php1 1/1 Running 0 6s 10.244.21.133 node-0001 <none> <none>
php2 1/1 Running 0 6s 10.244.147.9 node-0002 <none> <none>
php3 0/1 Pending 0 6s <none> <none> <none> <none>
- 模糊匹配策略
... ...
spec:
tolerations:
- operator: Exists // 部分匹配
key: k
effect: NoSchedule
... ...
- 容忍所有node污点
... ...
spec:
tolerations:
- operator: Exists // 模糊匹配
key: k // 键
effect: "" // 设置空或删除,代表所有污点标签
... ...
抢占与优先级
优先级概述
- 优先级是什么 - 优先级表示一个Pod相对于其他Pod的重要性
- 优先级有什么用? - 优先级可以保证重要的Pod被调度运行
- 如何使用优先级和抢占 - 配置优先级类PriorityClass- 创建Pod时为其设置对应的优先级
- PriorityClasss - PriorityClasss是一个全局资源对象,它定义了从优先级类名称到优先级整数值的映射- 优先级在value字段中指定,可以设置小于10亿的整数值,值最大,优先级最高
- PriorityClasss还有两个可选字段: - globalDefault用于设置默认优先级状态,如果没有任何优先级设置,Pod的优先级为零- description用来配置描述性信息,告诉用户优先级的用途
- 优先级策略 - 非抢占优先: - 指的是在调度阶段优先进行调度分配,一旦容器调度完成就不可以抢占,资源不足时,只能等待- 抢占优先: - 强制调度一个Pod,如果资源不足无法被调度,调度程序会抢占较低优先级的Pod的资源,来保证高优先级Pod的运行
非抢占优先级
- 定义优先级资源对象 - 创建一个value为1000的优先级对象- 创建一个value为500的优先级对象- 设置非抢占策略Never
[root@master ~]# vim mypriority.yaml
---
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
name: high-non // 指定优先级名字
preemptionPolicy: Never // 策略:非抢占
value: 1000 // 优先级
---
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
name: low-non
preemptionPolicy: Never
value: 500
[root@master ~]# kubectl apply -f mypriority.yaml
priorityclass.scheduling.k8s.io/high-non created
priorityclass.scheduling.k8s.io/low-non created
[root@master ~]# kubectl get priorityclasses.scheduling.k8s.io
NAME VALUE GLOBAL-DEFAULT AGE
high-non 1000 false 15s
low-non 500 false 15s
system-cluster-critical 2000000000 false 5d21h
system-node-critical 2000001000 false 5d21h
- 验证Pod优先级调度策略 - 所有Pod创建在node-0002上- 创建php1,该Pod使用默认优先级,cpu=1500m- 创建php1,该Pod使用低优先级,cpu=1500m- 创建php3,该Pod使用高优先级,cpu=1500m
root@master ~]# vim php1.yaml
---
kind: Pod
apiVersion: v1
metadata:
name: php1
spec:
nodeSelector:
kubernetes.io/hostname: node-0004
containers:
- name: php
image: myos:php-fpm
resources:
requests:
cpu: "1500m"
[root@master ~]# vim php2.yaml
---
kind: Pod
apiVersion: v1
metadata:
name: php2
spec:
nodeSelector:
kubernetes.io/hostname: node-0004
priorityClassName: low-non
containers:
- name: php
image: myos:php-fpm
resources:
requests:
cpu: "1500m"
[root@master ~]# vim php3.yaml
---
kind: Pod
apiVersion: v1
metadata:
name: php3
spec:
nodeSelector:
kubernetes.io/hostname: node-0004
priorityClassName: high-non
containers:
- name: php
image: myos:php-fpm
resources:
requests:
cpu: "1500m"
[root@master ~]# kubectl apply -f php1.yaml
pod/php1 created
[root@master ~]# kubectl apply -f php2.yaml
pod/php2 created
[root@master ~]# kubectl apply -f php3.yaml
pod/php3 created
[root@master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
php1 1/1 Running 0 19s
php2 0/1 Pending 0 16s
php3 0/1 Pending 0 14s
[root@master ~]# kubectl delete pod php1
pod "php1" deleted
[root@master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
php2 0/1 Pending 0 70s
php3 1/1 Running 0 68s
抢占优先级
- 定义优先级资源对象
[root@master ~]# vim mypriority.yaml
---
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
name: high-non
preemptionPolicy: Never
value: 1000
---
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
name: low-non
preemptionPolicy: Never
value: 500
---
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
name: high
preemptionPolicy: PreemptLowerPriority // 策略:抢占优先级
value: 1000
---
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
name: low
preemptionPolicy: PreemptLowerPriority
value: 500
[root@master ~]# kubectl apply -f mypriority.yaml
priorityclass.scheduling.k8s.io/high created
priorityclass.scheduling.k8s.io/low created
[root@master ~]# kubectl get priorityclasses.scheduling.k8s.io
NAME VALUE GLOBAL-DEFAULT AGE
high 1000 false 4s
high-non 1000 false 2h
low 500 false 4s
low-non 500 false 2h
system-cluster-critical 2000000000 false 21d
system-node-critical 2000001000 false 21d
- 验证抢占优先级
// 替换优先级策略
[root@master ~]# sed 's,-non,,' -i php?.yaml
// 默认优先级 Pod
[root@master ~]# kubectl apply -f php1.yaml
pod/php1 created
[root@master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
php1 1/1 Running 0 6s
// 高优先级 Pod
[root@master ~]# kubectl apply -f php3.yaml
pod/php3 created
[root@master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
php3 1/1 Running 0 9s
// 低优先级 Pod
[root@master ~]# kubectl apply -f php2.yaml
pod/php2 created
[root@master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
php2 0/1 Pending 0 3s
php3 1/1 Running 0 9s
Pod安全
特权容器
特权容器概述
- 什么是特权容器? - 容器是通过名称空间技术隔离的,有时候我们执行一些应用服务,需要使用或修改敏感的系统信息,这时容器需要突破隔离限制,获取更高的权限,这类容器统称特权容器- 运行特权容器会有一些安全风险,这种模式下运行容器对宿主机拥有root权限,可以突破隔离直接控制宿主机的资源配置
特权容器
- 制作特权容器(修改主机名和hosts文件)
[root@master ~]# vim root.yaml
---
kind: Pod
apiVersion: v1
metadata:
name: root
spec:
hostname: myhost // 修改容器主机名
hostAliases: // 修改 /etc/hosts
- ip: 192.168.1.30 // IP 地址
hostnames: // 名称键值对
- harbor // 主机名
containers:
- name: apache
image: myos:httpd
[root@master ~]# kubectl apply -f root.yaml
pod/root created
[root@master ~]# kubectl exec -it root -- /bin/bash
[root@myhost html]# hostname
myhost
[root@myhost html]# cat /etc/hosts
... ...
# Entries added by HostAliases.
192.168.1.30 harbor
- 制作root特权容器
[root@master ~]# vim root.yaml
---
kind: Pod
apiVersion: v1
metadata:
name: root
spec:
hostPID: true // 特权,共享系统进程
hostNetwork: true // 特权,共享主机网络
containers:
- name: apache
image: myos:httpd
securityContext: // 安全上下文值
privileged: true // root特权容器
[root@master ~]# kubectl apply -f root.yaml
pod/root created
[root@master ~]# kubectl exec -it root -- /bin/bash
[root@node-0005 html]# pstree -p // 系统进程特权
systemd(1)-+-NetworkManager(595)-+-{NetworkManager}(617)
| `-{NetworkManager}(618)
|-agetty(1129)
... ...
[root@node-0005 html]# ifconfig eth0 // 网络特权
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.55 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::f816:3eff:fe74:cd03 prefixlen 64 scopeid 0x20<link>
ether fa:16:3e:74:cd:03 txqueuelen 1000 (Ethernet)
[root@node-0005 ~]# mkdir /sysroot // root用户特权
[root@node-0005 ~]# mount /dev/sda1 /sysroot
[root@node-0005 ~]# chroot /sysroot
sh-4.4# // 此处已经是node节点上的root用户了
Pod安全策略
安全概述
- 什么是Pod安全策略? - Pod安全策略是集群级别的资源,它能够控制Pod运行的行为,以及它具有访问什么的能力
- 如何使用Pod安全策略? - Kubernetes服务器版本必须不低于v1.22- 确保PodSecurity特性门控被启用
- Pod安全策略(LEVEL) -
privieged
:不受限制的策略,提供最大可能范围的特权许可,此策略允许特权提升-baseline
:弱限制性的策略,禁止已知的策略提升权限,允许使用默认的Pod配置-restricted
:非常严格的限制性策略,遵循当前的保护Pod的最佳实践 - Pod准入控制标签(MODE) - Kubernetes定义了一组标签,你可以设置这些标签来定义某个名字空间上Pod安全性标准级别。所选择的标签定义了检测到潜在违利时所要采取的动作 -
enforce
:策略违利会导致Pod被拒绝-audit
:策略违利会触发审计日志,但是Pod仍可被接受-warn
:策略违利会触发用户可见的警告信息,但是Pod仍是被接受的 - 语法格式:
pod-security.kubernetes.io/<MODE>:<LEVEL>
限制特权容器
- 设置严格的准入控制,拒绝特权容器
[root@master ~]# kubectl create namespace myprod
namespace/myprod created
[root@master ~]# kubectl label namespaces myprod pod-security.kubernetes.io/enforce=restricted
namespace/myprod labeled
- 在创建特权容器时发出警告提示
[root@master ~]# kubectl create namespace mytest
namespace/mytest created
[root@master ~]# kubectl label namespaces mytest pod-security.kubernetes.io/warn=baseline
namespace/mytest labeled
- 创建特权容器
[root@master ~]# kubectl -n myprod apply -f root.yaml
Error from server (Failure): ... ... // 创建失败
[root@master ~]# kubectl -n mytest apply -f root.yaml
Warning: would violate "latest" version of "baseline" PodSecurity profile: host namespaces (hostNetwork=true, hostPID=true), privileged (container "linux" must not set securityContext.privileged=true)
pod/root created
- 创建安全的Pod
[root@master ~]# vim nonroot.yaml
---
kind: Pod
apiVersion: v1
metadata:
name: nonroot
spec:
restartPolicy: Always
containers:
- name: php
image: myos:php-fpm
securityContext:
allowPrivilegeEscalation: false
runAsNonRoot: true
runAsUser: 65534
seccompProfile:
type: "RuntimeDefault"
capabilities:
drop: ["ALL"]
[root@master ~]# kubectl -n myprod apply -f nonroot.yaml
pod/nonroot created
[root@master ~]# kubectl -n myprod get pods
NAME READY STATUS RESTARTS AGE
nonroot 1/1 Running 0 6s
[root@master ~]# kubectl -n myprod exec -it nonroot -- id // 没有root用户
uid=65534(nobody) gid=65534(nobody) groups=65534(nobody)
版权归原作者 川川野马 所有, 如有侵权,请联系我们删除。