kafka3.4.0版本升级--helm部署

kafka3.4.0版本升级–helm部署

前言

最近由于kafka的漏洞需要升级至3.4.0版本，之前由于不是helm部署，升级起来出现了权限问题、挂盘问题，在k8s搞了许久都搞不定，狠下心来，直接来一波helm安装，在2月份的时候，helm官网已推出chart-21.0.1包（https://artifacthub.io/packages/helm/bitnami/kafka）。用chart-21.0.1包准备开搞。

helm 安装kafka

helm 安装kafka比较简单，便不多说，官网有相应的说明，简单来说就是下载个chart包，在helm环境下执行helm安装即可。

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install my-kafka bitnami/kafka --version21.0.1

当然，这样安装是默认value.yaml的配置，但是对于实际应用，并非如此简单，我们要看懂实际部署的需求，比如，挂盘要求，service、端口等都是要根据实际生产或者说之前版本调整的。所以接下来要对value.yaml进行配置，配置属于自己“独一无二”的kafka。

“独一无二”的value.yaml

先说明下kafka实际要求：

1. 3个pod
2. storageClass为：xxx-xxx-xxx
3. 需要外界访问

修改默认的value.yaml部分配置：

1. 全局配置修改：

global:
  imageRegistry: ""## E.g.## imagePullSecrets:##   - myRegistryKeySecretName##
  imagePullSecrets: []
  storageClass: "xxx-xxx-xxx "

1. 修改镜像 这个比较简单，一般单位的服务器是没法直接连外网的，所以都会有自己的hub镜像仓库，所以先得从外网拉取镜像到自己仓库，然后将下列镜像改为自己的镜像。需要修改kakfa（bitnami/kafka:3.4.0-debian-11-r2）、zookeeper（zookeeper:3.8.1-debian-11-r0）、kubectl（kubectl:1.25.6-debian-11-r10）镜像：以kakfa修改为例：

image:
  registry: hubtest.xxx.com.cn
  repository: xxx/
  tag: bitnami/kafka:3.4.0-debian-11-r2
  digest: ""#无需填

1. 设置副本数

replicaCount: 3

1. 补充资源配置 一定要在resources这一步补充资源配置，不然pod是启动不了的，那就蛋疼了。

resources:
  requests:
    memory: "64Mi"
    cpu: "250m"
  limits:
    memory: "1280Mi"
    cpu: "500m"

1. 启动集群外部访问kafka–开启externalAccess 这一步非常关键，因为一般应用日志会往kafka上吐，这就是给应用或者filebeat提供吐的入口。做了以下修改： externalAccess中enabled 设置为 true，autoDiscovery:设置为true kubectl镜像设置为本地仓库镜像 service type设置为： NodePort （因为有F5作为负载均衡，所以就不用k8s service自带的LoadBalancer，只要求暴露端口就行，所以就采用nodeport方式。） 添加resource资源 nodePorts: - 30001- 30002- 30003 注意：个数要和副本数（3）一样 有个巨坑得和大家说说说，配置中有这么一句话Note: RBAC might be required 也就是当你启动externalAccess，启动helm的要带上一个参数，不然会报关于RBAC的错误，加上以下一行就行。 –set rbac.create=true 如果是rancher 自带的helm可视化部署，直接在命令那一行输入rbac.create=true 访问方式： :9094 或者:30001 具体修改如下：

## External Access to Kafka brokers configuration##
externalAccess:
  ## @param externalAccess.enabled Enable Kubernetes external cluster access to Kafka brokers##
  enabled: true## External IPs auto-discovery configuration## An init container is used to auto-detect LB IPs or node ports by querying the K8s API## Note: RBAC might be required##
  autoDiscovery:
    ## @param externalAccess.autoDiscovery.enabled Enable using an init container to auto-detect external IPs/ports by querying the K8s API##
    enabled: true## Bitnami Kubectl image## ref: https://hub.docker.com/r/bitnami/kubectl/tags/## @param externalAccess.autoDiscovery.image.registry Init container auto-discovery image registry## @param externalAccess.autoDiscovery.image.repository Init container auto-discovery image repository## @param externalAccess.autoDiscovery.image.tag Init container auto-discovery image tag (immutable tags are recommended)## @param externalAccess.autoDiscovery.image.digest Kubectl image digest in the way sha256:aa.... Please note this parameter, if set, will override the tag## @param externalAccess.autoDiscovery.image.pullPolicy Init container auto-discovery image pull policy## @param externalAccess.autoDiscovery.image.pullSecrets Init container auto-discovery image pull secrets##
    image:
      registry: hubtest.xxx.com.cn
      repository: bitnami/kubectl
      tag: 1.25.6-debian-11-r10
      digest: ""## Specify a imagePullPolicy## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'## ref: https://kubernetes.io/docs/user-guide/images/#pre-pulling-images##
      pullPolicy: IfNotPresent
      ## Optionally specify an array of imagePullSecrets (secrets must be manually created in the namespace)## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/## e.g:## pullSecrets:##   - myRegistryKeySecretName##
      pullSecrets: []## Init Container resource requests and limits## ref: https://kubernetes.io/docs/user-guide/compute-resources/## @param externalAccess.autoDiscovery.resources.limits The resources limits for the auto-discovery init container## @param externalAccess.autoDiscovery.resources.requests The requested resources for the auto-discovery init container##
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "1280Mi"
        cpu: "500m"## Parameters to configure K8s service(s) used to externally access Kafka brokers## Note: A new service per broker will be created##
  service:
    ## @param externalAccess.service.type Kubernetes Service type for external access. It can be NodePort, LoadBalancer or ClusterIP##
    type: NodePort
    ## @param externalAccess.service.ports.external Kafka port used for external access when service type is LoadBalancer##
    ports:
      external: 9094## @param externalAccess.service.loadBalancerIPs Array of load balancer IPs for each Kafka broker. Length must be the same as replicaCount## e.g:## loadBalancerIPs:##   - X.X.X.X##   - Y.Y.Y.Y##
    loadBalancerIPs: []## @param externalAccess.service.loadBalancerNames Array of load balancer Names for each Kafka broker. Length must be the same as replicaCount## e.g:## loadBalancerNames:##   - broker1.external.example.com##   - broker2.external.example.com##
    loadBalancerNames: []## @param externalAccess.service.loadBalancerAnnotations Array of load balancer annotations for each Kafka broker. Length must be the same as replicaCount## e.g:## loadBalancerAnnotations:##   - external-dns.alpha.kubernetes.io/hostname: broker1.external.example.com.##   - external-dns.alpha.kubernetes.io/hostname: broker2.external.example.com.##
    loadBalancerAnnotations: []## @param externalAccess.service.loadBalancerSourceRanges Address(es) that are allowed when service is LoadBalancer## ref: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/#restrict-access-for-loadbalancer-service## e.g:## loadBalancerSourceRanges:## - 10.10.10.0/24##
    loadBalancerSourceRanges: []## @param externalAccess.service.nodePorts Array of node ports used for each Kafka broker. Length must be the same as replicaCount## e.g:## nodePorts:##   - 30001##   - 30002##
    nodePorts: 
      - 30001
      - 30002
      - 30003## @param externalAccess.service.useHostIPs Use service host IPs to configure Kafka external listener when service type is NodePort##
    useHostIPs: false## @param externalAccess.service.usePodIPs using the MY_POD_IP address for external access.##
    usePodIPs: false## @param externalAccess.service.domain Domain or external ip used to configure Kafka external listener when service type is NodePort or ClusterIP## NodePort: If not specified, the container will try to get the kubernetes node external IP## ClusterIP: Must be specified, ingress IP or domain where tcp for external ports is configured##
    domain: ""## @param externalAccess.service.publishNotReadyAddresses Indicates that any agent which deals with endpoints for this Service should disregard any indications of ready/not-ready## ref: https://kubernetes.io/docs/reference/kubernetes-api/service-resources/service-v1/
    publishNotReadyAddresses: false## @param externalAccess.service.labels Service labels for external access##
    labels: {}## @param externalAccess.service.annotations Service annotations for external access##
    annotations: {}## @param externalAccess.service.extraPorts Extra ports to expose in the Kafka external service##
    extraPorts: []

参数详细见官网和一个不错的文章：
https://artifacthub.io/packages/helm/bitnami/kafka https://www.cnblogs.com/east4ming/p/17017779.html