0


云端技术驾驭DAY13——Pod污点、容忍策略、Pod优先级与抢占、容器安全

往期回顾:
云端技术驾驭DAY01——云计算底层技术奥秘、云服务器磁盘技术、虚拟化管理、公有云概述
云端技术驾驭DAY02——华为云管理、云主机管理、跳板机配置、制作私有镜像模板
云端技术驾驭DAY03——云主机网站部署、web集群部署、Elasticsearch安装
云端技术驾驭DAY04——Logstash安装部署及插件模块
云端技术驾驭DAY06——容器技术概述、镜像与容器管理、定制简单镜像、容器内安装部署服务
云端技术驾驭DAY07——Dockerfile详解、容器镜像制作、私有仓库
云端技术驾驭DAY08——部署容器服务、Compose微服务管理、harbor仓库部署及管理
云端技术驾驭DAY09——k8s集群安装部署、calico插件部署、计算节点配置管理
云端技术驾驭DAY10——kubectl命令详解、Pod创建过程、Pod的生命周期、定制Pod、资源对象文件
云端技术驾驭DAY11——资源对象文件、Pod自定义命令、多容器Pod、资源监控工具
云端技术驾驭DAY12——Pod调度策略、Pod标签管理、Pod资源配额与限额、全局资源配额与限额策略

云端技术驾驭DAY13

Pod调度策略管理

污点容忍策略

污点概述
  • 什么是污点? - 污点(Taint)是使节点与Pod产生排斥的一类规则
  • 污点策略是如何实现 - 污点策略通过嵌合在键值对上的污点标签进行声明
  • 污点标签 - 尽量不调度:PreferNoSchedule- 不会被调度:NoSchedule- 驱逐节点:NoExecute
  • 管理污点标签 - 污点标签必须绑定在键值对上,格式为:key=value:污点标签- 查看污点标签:kubectl describe nodes [节点名字]- 设置污点标签:kubectl taint node [节点名字] key=value:污点标签- 删除污点标签:kubectl taint node [节点名字] key=value:污点标签-
污点策略
  • 污点验证 - 为node-0001设置PreferNoschedule污点标签- 为node-0002设置NoSchedule污点标签
[root@master ~]# kubectl taint node node-0001 k=v1:PreferNoSchedule
node/node-0001 tainted
[root@master ~]# kubectl taint node node-0002 k=v2:NoSchedule
node/node-0002 tainted
[root@master ~]# kubectl describe nodes | grep Taints
Taints:             node-role.kubernetes.io/control-plane:NoSchedule
Taints:             k=v1:PreferNoSchedule
Taints:             k=v2:NoSchedule
Taints:             <none>
Taints:             <none>
Taints:             <none>
  • Pod资源文件
[root@master ~]# vim myphp.yaml
---
kind: Pod
apiVersion: v1
metadata:
  name: myphp
spec:
  containers:
  - name: php
    image: myos:php-fpm
    resources:
      requests:
        cpu: 1500m
  • 验证测试
[root@master ~]# sed "s,myphp,php1," myphp.yaml | kubectl apply -f -
pod/php1 created
[root@master ~]# sed "s,myphp,php2," myphp.yaml | kubectl apply -f -
pod/php2 created
[root@master ~]# sed "s,myphp,php3," myphp.yaml | kubectl apply -f -          // 创建三个Pod
pod/php3 created
[root@master ~]# kubectl get pods -o wide                                     // 发现会优先使用没有污点的节点
NAME   READY   STATUS    RESTARTS   AGE   IP               NODE        NOMINATED NODE   READINESS GATES
php1   1/1     Running   0          35s   10.244.240.134   node-0004   <none>           <none>
php2   1/1     Running   0          30s   10.244.243.209   node-0003   <none>           <none>
php3   1/1     Running   0          26s   10.244.153.139   node-0005   <none>           <none>
  • 在没有其他节点可用的时候,使用PreferNoSchedule污点所在的节点运行Pod
[root@master ~]# sed "s,myphp,php4," myphp.yaml | kubectl apply -f -
pod/php4 created
[root@master ~]# kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE     IP               NODE        NOMINATED NODE   READINESS GATES
php1   1/1     Running   0          4m19s   10.244.240.134   node-0004   <none>           <none>
php2   1/1     Running   0          4m14s   10.244.243.209   node-0003   <none>           <none>
php3   1/1     Running   0          4m10s   10.244.153.139   node-0005   <none>           <none>
php4   1/1     Running   0          2s      10.244.21.132    node-0001   <none>           <none>
  • 继续创建Pod - 即使Pod创建失败,也不会使用NoSchedule所在节点
[root@master ~]# sed "s,myphp,php5," myphp.yaml | kubectl apply -f -
pod/php5 created
[root@master ~]# kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE     IP               NODE        NOMINATED NODE   READINESS GATES
php1   1/1     Running   0          5m37s   10.244.240.134   node-0004   <none>           <none>
php2   1/1     Running   0          5m32s   10.244.243.209   node-0003   <none>           <none>
php3   1/1     Running   0          5m28s   10.244.153.139   node-0005   <none>           <none>
php4   1/1     Running   0          80s     10.244.21.132    node-0001   <none>           <none>
php5   0/1     Pending   0          2s      <none>           <none>      <none>           <none>
  • NoSchedule策略 - 设置NoSchedule污点标签只对新建Pod有效,对于已经创建完成的Pod不会产生影响
[root@master ~]# kubectl taint node node-0003 k=v3:NoSchedule
node/node-0003 tainted
[root@master ~]# kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE     IP               NODE        NOMINATED NODE   READINESS GATES
php1   1/1     Running   0          8m38s   10.244.240.134   node-0004   <none>           <none>
php2   1/1     Running   0          8m33s   10.244.243.209   node-0003   <none>           <none>
php3   1/1     Running   0          8m29s   10.244.153.139   node-0005   <none>           <none>
php4   1/1     Running   0          4m21s   10.244.21.132    node-0001   <none>           <none>
php5   0/1     Pending   0          3m3s    <none>           <none>      <none>           <none>
  • 驱逐策略 - 驱逐策略会删除该节点上的所有Pod
[root@master ~]# kubectl taint node node-0004 k=v4:NoExecute             // 为node-0004设置NoExecute策略
node/node-0004 tainted
[root@master ~]# kubectl describe nodes | grep Taints
Taints:             node-role.kubernetes.io/control-plane:NoSchedule
Taints:             k=v1:PreferNoSchedule
Taints:             k=v2:NoSchedule
Taints:             k=v3:NoSchedule
Taints:             k=v4:NoExecute
Taints:             <none>
[root@master ~]# kubectl get pods -o wide                               // 查看Pod情况,NoExecute污点所在节点的Pod已经被删除了
NAME   READY   STATUS    RESTARTS   AGE     IP               NODE        NOMINATED NODE   READINESS GATES
php2   1/1     Running   0          15m     10.244.243.209   node-0003   <none>           <none>
php3   1/1     Running   0          15m     10.244.153.139   node-0005   <none>           <none>
php4   1/1     Running   0          11m     10.244.21.132    node-0001   <none>           <none>
php5   0/1     Pending   0          9m57s   <none>           <none>      <none>           <none>
  • 删除实验所有污点策略
[root@master ~]# kubectl taint node node-000{1..4} k-
node/node-0001 untainted
node/node-0002 untainted
node/node-0003 untainted
node/node-0004 untainted
容忍策略
  • 容忍策略是什么? - 容忍策略与污点策略相反,某些时候我们需要在有污点的节点上运行Pod,这种无视污点标签的调度方式称为容忍策略
  • 容忍策略实验 - 节点node-000{1…2}设置污点标签k=v1:NoSchedule- 节点node-000{3…4}设置污点标签k=v2:NoSchedule- 节点node-0005设置污点标签k=v1:NoExecute
  • 精确匹配策略
[root@master ~]# kubectl taint node node-000{1..2} k=v1:NoSchedule
node/node-0001 tainted
node/node-0002 tainted
[root@master ~]# kubectl taint node node-000{3..4} k=v2:NoSchedule
node/node-0003 tainted
node/node-0004 tainted
[root@master ~]# kubectl taint node node-0005 k=v1:NoExecute
node/node-0005 tainted
[root@master ~]# vim myphp.yaml              // 容忍k=v1:NoSchedule污点
---
kind: Pod
apiVersion: v1
metadata:
  name: myphp
spec:
  tolerations:
  - operator: Equal                         // 完全匹配键值对
    key: k                                  // 键
    value: v1                               // 值
    effect: NoSchedule                      // 污点标签
  containers:
  - name: php
    image: myos:php-fpm
    resources:
      requests:
        cpu: 1500m
[root@master ~]# for i in php{1..3};do sed "s,myphp,${i}," myphp.yaml ;done|kubectl apply -f -
pod/php1 created
pod/php2 created
pod/php3 created
[root@master ~]# kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP              NODE        NOMINATED NODE   READINESS GATES
php1   1/1     Running   0          6s    10.244.21.133   node-0001   <none>           <none>
php2   1/1     Running   0          6s    10.244.147.9    node-0002   <none>           <none>
php3   0/1     Pending   0          6s    <none>          <none>      <none>           <none>
  • 模糊匹配策略
... ...
spec:
  tolerations:
  - operator: Exists            // 部分匹配
    key: k
    effect: NoSchedule
... ...
  • 容忍所有node污点
... ...
spec:
  tolerations:
  - operator: Exists     // 模糊匹配
    key: k               // 键
    effect: ""           // 设置空或删除,代表所有污点标签
... ...

抢占与优先级

优先级概述
  • 优先级是什么 - 优先级表示一个Pod相对于其他Pod的重要性
  • 优先级有什么用? - 优先级可以保证重要的Pod被调度运行
  • 如何使用优先级和抢占 - 配置优先级类PriorityClass- 创建Pod时为其设置对应的优先级
  • PriorityClasss - PriorityClasss是一个全局资源对象,它定义了从优先级类名称到优先级整数值的映射- 优先级在value字段中指定,可以设置小于10亿的整数值,值最大,优先级最高
  • PriorityClasss还有两个可选字段: - globalDefault用于设置默认优先级状态,如果没有任何优先级设置,Pod的优先级为零- description用来配置描述性信息,告诉用户优先级的用途
  • 优先级策略 - 非抢占优先: - 指的是在调度阶段优先进行调度分配,一旦容器调度完成就不可以抢占,资源不足时,只能等待- 抢占优先: - 强制调度一个Pod,如果资源不足无法被调度,调度程序会抢占较低优先级的Pod的资源,来保证高优先级Pod的运行
非抢占优先级
  • 定义优先级资源对象 - 创建一个value为1000的优先级对象- 创建一个value为500的优先级对象- 设置非抢占策略Never
[root@master ~]# vim mypriority.yaml
---
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
  name: high-non                    // 指定优先级名字
preemptionPolicy: Never             // 策略:非抢占
value: 1000                         // 优先级

---
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
  name: low-non
preemptionPolicy: Never
value: 500
[root@master ~]# kubectl apply -f mypriority.yaml 
priorityclass.scheduling.k8s.io/high-non created
priorityclass.scheduling.k8s.io/low-non created
[root@master ~]# kubectl get priorityclasses.scheduling.k8s.io 
NAME                      VALUE        GLOBAL-DEFAULT   AGE
high-non                  1000         false            15s
low-non                   500          false            15s
system-cluster-critical   2000000000   false            5d21h
system-node-critical      2000001000   false            5d21h
  • 验证Pod优先级调度策略 - 所有Pod创建在node-0002上- 创建php1,该Pod使用默认优先级,cpu=1500m- 创建php1,该Pod使用低优先级,cpu=1500m- 创建php3,该Pod使用高优先级,cpu=1500m
root@master ~]# vim php1.yaml
--- 
kind: Pod
apiVersion: v1
metadata:
  name: php1
spec:
  nodeSelector:
    kubernetes.io/hostname: node-0004
  containers:
  - name: php
    image: myos:php-fpm
    resources:
      requests:
        cpu: "1500m"
[root@master ~]# vim php2.yaml 
---
kind: Pod
apiVersion: v1
metadata:
  name: php2
spec:
  nodeSelector:
    kubernetes.io/hostname: node-0004
  priorityClassName: low-non
  containers:
  - name: php
    image: myos:php-fpm
    resources:
      requests:
        cpu: "1500m"
[root@master ~]# vim php3.yaml 
---
kind: Pod
apiVersion: v1
metadata:
  name: php3
spec:
  nodeSelector:
    kubernetes.io/hostname: node-0004
  priorityClassName: high-non    
  containers:
  - name: php
    image: myos:php-fpm
    resources:
      requests:
        cpu: "1500m"
[root@master ~]# kubectl apply -f php1.yaml 
pod/php1 created
[root@master ~]# kubectl apply -f php2.yaml 
pod/php2 created
[root@master ~]# kubectl apply -f php3.yaml 
pod/php3 created
[root@master ~]# kubectl get pods
NAME   READY   STATUS    RESTARTS   AGE
php1   1/1     Running   0          19s
php2   0/1     Pending   0          16s
php3   0/1     Pending   0          14s
[root@master ~]# kubectl delete pod php1
pod "php1" deleted
[root@master ~]# kubectl get pods
NAME   READY   STATUS    RESTARTS   AGE
php2   0/1     Pending   0          70s
php3   1/1     Running   0          68s
抢占优先级
  • 定义优先级资源对象
[root@master ~]# vim mypriority.yaml 
---
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
  name: high-non
preemptionPolicy: Never
value: 1000

---
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
  name: low-non
preemptionPolicy: Never
value: 500

---
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
  name: high
preemptionPolicy: PreemptLowerPriority              // 策略:抢占优先级
value: 1000

---
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
  name: low
preemptionPolicy: PreemptLowerPriority
value: 500
[root@master ~]# kubectl apply -f mypriority.yaml 
priorityclass.scheduling.k8s.io/high created
priorityclass.scheduling.k8s.io/low created
[root@master ~]# kubectl get priorityclasses.scheduling.k8s.io  
NAME                      VALUE        GLOBAL-DEFAULT   AGE
high                      1000         false            4s
high-non                  1000         false            2h
low                       500          false            4s
low-non                   500          false            2h
system-cluster-critical   2000000000   false            21d
system-node-critical      2000001000   false            21d
  • 验证抢占优先级
// 替换优先级策略
[root@master ~]# sed 's,-non,,' -i php?.yaml

// 默认优先级 Pod
[root@master ~]# kubectl apply -f php1.yaml 
pod/php1 created
[root@master ~]# kubectl get pods
NAME   READY   STATUS    RESTARTS   AGE
php1   1/1     Running   0          6s

// 高优先级 Pod
[root@master ~]# kubectl apply -f php3.yaml
pod/php3 created
[root@master ~]# kubectl get pods
NAME   READY   STATUS    RESTARTS   AGE
php3   1/1     Running   0          9s

// 低优先级 Pod
[root@master ~]# kubectl apply -f php2.yaml
pod/php2 created
[root@master ~]# kubectl get pods
NAME   READY   STATUS    RESTARTS   AGE
php2   0/1     Pending   0          3s
php3   1/1     Running   0          9s

Pod安全

特权容器

特权容器概述
  • 什么是特权容器? - 容器是通过名称空间技术隔离的,有时候我们执行一些应用服务,需要使用或修改敏感的系统信息,这时容器需要突破隔离限制,获取更高的权限,这类容器统称特权容器- 运行特权容器会有一些安全风险,这种模式下运行容器对宿主机拥有root权限,可以突破隔离直接控制宿主机的资源配置
特权容器
  • 制作特权容器(修改主机名和hosts文件)
[root@master ~]# vim root.yaml
---
kind: Pod
apiVersion: v1
metadata:
  name: root
spec:
  hostname: myhost         // 修改容器主机名
  hostAliases:             // 修改 /etc/hosts
  - ip: 192.168.1.30       // IP 地址
    hostnames:             // 名称键值对
    - harbor               // 主机名
  containers:
  - name: apache
    image: myos:httpd
[root@master ~]# kubectl apply -f root.yaml 
pod/root created
[root@master ~]# kubectl exec -it root -- /bin/bash
[root@myhost html]# hostname
myhost
[root@myhost html]# cat /etc/hosts
... ...
# Entries added by HostAliases.
192.168.1.30    harbor
  • 制作root特权容器
[root@master ~]# vim root.yaml
---
kind: Pod
apiVersion: v1
metadata:
  name: root
spec:
  hostPID: true            // 特权,共享系统进程
  hostNetwork: true        // 特权,共享主机网络
  containers:
  - name: apache
    image: myos:httpd
    securityContext:       // 安全上下文值
      privileged: true     // root特权容器
[root@master ~]# kubectl apply -f root.yaml 
pod/root created
[root@master ~]# kubectl exec -it root -- /bin/bash
[root@node-0005 html]# pstree -p                              // 系统进程特权
systemd(1)-+-NetworkManager(595)-+-{NetworkManager}(617)
           |                     `-{NetworkManager}(618)
           |-agetty(1129)
           ... ...
[root@node-0005 html]# ifconfig eth0                          // 网络特权
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.55  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::f816:3eff:fe74:cd03  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:74:cd:03  txqueuelen 1000  (Ethernet)
[root@node-0005 ~]# mkdir /sysroot                            // root用户特权
[root@node-0005 ~]# mount /dev/sda1 /sysroot
[root@node-0005 ~]# chroot /sysroot
sh-4.4#                                                       // 此处已经是node节点上的root用户了

Pod安全策略

安全概述
  • 什么是Pod安全策略? - Pod安全策略是集群级别的资源,它能够控制Pod运行的行为,以及它具有访问什么的能力
  • 如何使用Pod安全策略? - Kubernetes服务器版本必须不低于v1.22- 确保PodSecurity特性门控被启用
  • Pod安全策略(LEVEL) - privieged:不受限制的策略,提供最大可能范围的特权许可,此策略允许特权提升- baseline:弱限制性的策略,禁止已知的策略提升权限,允许使用默认的Pod配置- restricted:非常严格的限制性策略,遵循当前的保护Pod的最佳实践
  • Pod准入控制标签(MODE) - Kubernetes定义了一组标签,你可以设置这些标签来定义某个名字空间上Pod安全性标准级别。所选择的标签定义了检测到潜在违利时所要采取的动作 - enforce:策略违利会导致Pod被拒绝- audit:策略违利会触发审计日志,但是Pod仍可被接受- warn:策略违利会触发用户可见的警告信息,但是Pod仍是被接受的
  • 语法格式:pod-security.kubernetes.io/<MODE>:<LEVEL>
限制特权容器
  • 设置严格的准入控制,拒绝特权容器
[root@master ~]# kubectl create namespace myprod
namespace/myprod created
[root@master ~]# kubectl label namespaces myprod pod-security.kubernetes.io/enforce=restricted
namespace/myprod labeled
  • 在创建特权容器时发出警告提示
[root@master ~]# kubectl create namespace mytest
namespace/mytest created
[root@master ~]# kubectl label namespaces mytest pod-security.kubernetes.io/warn=baseline
namespace/mytest labeled
  • 创建特权容器
[root@master ~]# kubectl -n myprod apply -f root.yaml 
Error from server (Failure): ... ...                                // 创建失败
[root@master ~]# kubectl -n mytest apply -f root.yaml                                    
Warning: would violate "latest" version of "baseline" PodSecurity profile: host namespaces (hostNetwork=true, hostPID=true), privileged (container "linux" must not set securityContext.privileged=true)
pod/root created
  • 创建安全的Pod
[root@master ~]# vim nonroot.yaml
---
kind: Pod
apiVersion: v1
metadata:
  name: nonroot
spec:
  restartPolicy: Always
  containers:
  - name: php
    image: myos:php-fpm
    securityContext:
      allowPrivilegeEscalation: false
      runAsNonRoot: true
      runAsUser: 65534
      seccompProfile:
        type: "RuntimeDefault"
      capabilities:
        drop: ["ALL"]
[root@master ~]# kubectl -n myprod apply -f nonroot.yaml 
pod/nonroot created
[root@master ~]# kubectl -n myprod get pods
NAME      READY   STATUS    RESTARTS   AGE
nonroot   1/1     Running   0          6s
[root@master ~]# kubectl -n myprod exec -it nonroot -- id           // 没有root用户
uid=65534(nobody) gid=65534(nobody) groups=65534(nobody)
标签: 安全 微服务 运维

本文转载自: https://blog.csdn.net/2301_78214375/article/details/136313134
版权归原作者 川川野马 所有, 如有侵权,请联系我们删除。

“云端技术驾驭DAY13——Pod污点、容忍策略、Pod优先级与抢占、容器安全”的评论:

还没有评论