（二）Kafka 安全之使用 SSL 的加密和身份验证

目录

一. 前言

接上一篇《（一）Kafka 安全之使用 SSL 的加密和身份验证》，本文从 2.2 小节开始。

二. 使用 SSL 的加密和身份验证

2.2. 创建你自己的 CA（Creating your own CA）

原文引用：After this step each machine in the cluster has a public/private key pair which can already be used to encrypt traffic and a certificate signing request, which is the basis for creating a certificate. To add authentication capabilities this signing request needs to be signed by a trusted authority, which will be created in this step.

在这一步骤之后，集群中的每台机器都有一个公钥/私钥对，该对已经可以用于加密传输和证书签名请求，这是创建证书的基础。若要添加身份验证功能，此签名请求需要由将在该步骤中创建的可信机构进行签名。

原文引用：A certificate authority (CA) is responsible for signing certificates. CAs works likes a government that issues passports - the government stamps (signs) each passport so that the passport becomes difficult to forge. Other governments verify the stamps to ensure the passport is authentic. Similarly, the CA signs the certificates, and the cryptography guarantees that a signed certificate is computationally difficult to forge. Thus, as long as the CA is a genuine and trusted authority, the clients have a strong assurance that they are connecting to the authentic machines.

证书颁发机构（CA）负责对证书进行签名。CA 的运作方式就像一个签发护照的政府——政府在每本护照上盖章（签字），这样护照就很难伪造。其他政府则会核实印章，以确保护照的真实性。同样，CA 对证书进行签名，加密技术保证签名的证书在计算上很难伪造。因此，只要 CA 是真实可信的授权机构，客户端就可以有力地保证他们连接到了真实的机器。

原文引用：For this guide we will be our own Certificate Authority. When setting up a production cluster in a corporate environment these certificates would usually be signed by a corporate CA that is trusted throughout the company. Please see Common Pitfalls in Production for some things to consider for this case.

对于本指南，我们将成为我们自己的证书颁发机构。在公司环境中设置生产集群时，这些证书通常由整个公司都信任的公司 CA 签名。请参阅生产中的常见缺陷，了解本案例中需要考虑的一些事项。

原文引用：Due to a bug in OpenSSL, the x509 module will not copy requested extension fields from CSRs into the final certificate. Since we want the SAN extension to be present in our certificate to enable hostname verification, we'll use the ca module instead. This requires some additional configuration to be in place before we generate our CA keypair.

Save the following listing into a file called openssl-ca.cnf and adjust the values for validity and common attributes as necessary.

由于 OpenSSL 中的一个 Bug，x509 模块不会将请求的扩展字段从 CSRs 复制到最终证书中。由于我们希望 SAN 扩展出现在我们的证书中以启用主机名验证，因此我们将使用 CA 模块。这需要在我们生成 CA 密钥对之前进行一些额外的配置。

将以下列表保存到名为 openssl-ca.cnf 的文件中，并根据需要调整有效性和通用属性的值。

HOME            = .
RANDFILE        = $ENV::HOME/.rnd

####################################################################
[ ca ]
default_ca    = CA_default      # The default ca section

[ CA_default ]

base_dir      = .
certificate   = $base_dir/cacert.pem   # The CA certificate
private_key   = $base_dir/cakey.pem    # The CA private key
new_certs_dir = $base_dir              # Location for new certs after signing
database      = $base_dir/index.txt    # Database index file
serial        = $base_dir/serial.txt   # The current serial number

default_days     = 1000         # How long to certify for
default_crl_days = 30           # How long before next CRL
default_md       = sha256       # Use public key default MD
preserve         = no           # Keep passed DN ordering

x509_extensions = ca_extensions # The extensions to add to the cert

email_in_dn     = no            # Don't concat the email in the DN
copy_extensions = copy          # Required to copy SANs from CSR to cert

####################################################################
[ req ]
default_bits       = 4096
default_keyfile    = cakey.pem
distinguished_name = ca_distinguished_name
x509_extensions    = ca_extensions
string_mask        = utf8only

####################################################################
[ ca_distinguished_name ]
countryName         = Country Name (2 letter code)
countryName_default = DE

stateOrProvinceName         = State or Province Name (full name)
stateOrProvinceName_default = Test Province

localityName                = Locality Name (eg, city)
localityName_default        = Test Town

organizationName            = Organization Name (eg, company)
organizationName_default    = Test Company

organizationalUnitName         = Organizational Unit (eg, division)
organizationalUnitName_default = Test Unit

commonName         = Common Name (e.g. server FQDN or YOUR name)
commonName_default = Test Name

emailAddress         = Email Address
emailAddress_default = [email protected]

####################################################################
[ ca_extensions ]

subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid:always, issuer
basicConstraints       = critical, CA:true
keyUsage               = keyCertSign, cRLSign

####################################################################
[ signing_policy ]
countryName            = optional
stateOrProvinceName    = optional
localityName           = optional
organizationName       = optional
organizationalUnitName = optional
commonName             = supplied
emailAddress           = optional

####################################################################
[ signing_req ]
subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid,issuer
basicConstraints       = CA:FALSE
keyUsage               = digitalSignature, keyEncipherment

原文引用：Then create a database and serial number file, these will be used to keep track of which certificates were signed with this CA. Both of these are simply text files that reside in the same directory as your CA keys.

然后创建一个数据库和序列号文件，这些文件将用于跟踪使用此 CA 签名的证书。这两个文件都只是文本文件，与 CA 密钥位于同一目录中。

> echo 01 > serial.txt
> touch index.txt

原文引用：With these steps done you are now ready to generate your CA that will be used to sign certificates later.

完成这些步骤后，您现在可以生成 CA，该 CA 稍后将用于签署证书。

> openssl req -x509 -config openssl-ca.cnf -newkey rsa:4096 -sha256 -nodes -out cacert.pem -outform PEM

原文引用：The CA is simply a public/private key pair and certificate that is signed by itself, and is only intended to sign other certificates.

This keypair should be kept very safe, if someone gains access to it, they can create and sign certificates that will be trusted by your infrastructure, which means they will be able to impersonate anybody when connecting to any service that trusts this CA.

The next step is to add the generated CA to the clients' truststore so that the clients can trust this CA:

CA 只是一个由自己签名的公钥/私钥对和证书，仅用于对其他证书进行签名。

这个密钥对应该非常安全，如果有人访问了它，他们可以创建并签署您的基础设施信任的证书，这意味着他们在连接到信任这个 CA 的任何服务时都可以模拟任何人。

下一步是将生成的 CA 添加到 **clients' truststore** 中，以便客户端可以信任此 CA：

> keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert

原文引用：Note: If you configure the Kafka brokers to require client authentication by setting ssl.client.auth to be "requested" or "required" in the Kafka brokers config then you must provide a truststore for the Kafka brokers as well and it should have all the CA certificates that clients' keys were signed by.

注意：如果您通过在 Kafka brokers config 中将 ssl.client.auth 设置为“requested”或“required”来配置 Kafka brokers 以要求客户端身份验证，那么您也必须为 Kafka Broker 提供一个信任库，并且它应该具有客户端密钥签名的所有 CA 证书。

> keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert

原文引用：In contrast to the keystore in step 1 that stores each machine's own identity, the truststore of a client stores all the certificates that the client should trust. Importing a certificate into one's truststore also means trusting all certificates that are signed by that certificate. As the analogy above, trusting the government (CA) also means trusting all passports (certificates) that it has issued. This attribute is called the chain of trust, and it is particularly useful when deploying SSL on a large Kafka cluster. You can sign all certificates in the cluster with a single CA, and have all machines share the same truststore that trusts the CA. That way all machines can authenticate all other machines.

与步骤1中存储每台机器自己身份的密钥库不同，客户端的信任库存储客户端应该信任的所有证书。将证书导入自己的信任库也就意味着信任由该证书签名的所有证书。如上所述，信任政府（CA）也就意味着信任政府颁发的所有护照（证书）。这个属性被称为信任链，当在大型 Kafka集群上部署 SSL 时，它特别有用。您可以使用单个 CA 对集群中的所有证书进行签名，并使所有计算机共享信任该 CA 的同一信任库。这样，所有计算机都可以对所有其他计算机进行身份验证。

2.3. 签署证书（Signing the certificate）

然后与 CA 签署（Then sign it with the CA）：

> openssl ca -config openssl-ca.cnf -policy signing_policy -extensions signing_req -out {server certificate} -infiles {certificate signing request}

原文引用：Finally, you need to import both the certificate of the CA and the signed certificate into the keystore:

最后，您需要将 CA 的证书和已签名的证书导入密钥库：

> keytool -keystore {keystore} -alias CARoot -import -file {CA certificate}
> keytool -keystore {keystore} -alias localhost -import -file cert-signed

原文引用：The definitions of the parameters are the following:

1. keystore: the location of the keystore
2. CA certificate: the certificate of the CA
3. certificate signing request: the csr created with the server key
4. server certificate: the file to write the signed certificate of the server to

参数的定义如下：

1. keystore：密钥库的位置。
2. CA certificate：CA 的证书。
3. 证书签名请求：使用服务器密钥创建的 csr。
4. 服务器证书：将服务器的签名证书写入的文件。

原文引用：This will leave you with one truststore called truststore.jks - this can be the same for all clients and brokers and does not contain any sensitive information, so there is no need to secure this.

这将给您留下一个名为 truststore.jks 的信任库 - 这对所有客户端和 Broker 都是一样的，并且不包含任何敏感信息，因此没有必要对此进行安全保护。

原文引用：Additionally you will have one server.keystore.jks file per node which contains that nodes keys, certificate and your CAs certificate, please refer to Configuring Kafka Brokers and Configuring Kafka Clients for information on how to use these files.

For some tooling assistance on this topic, please check out the easyRSA project which has extensive scripting in place to help with these steps.

此外，每个节点将有一个 server.keystore.jks 文件，其中包含该节点的密钥、证书和您的 CA 证书，有关如何使用这些文件的信息，请参阅配置 Kafka Brokers 和配置 Kafka Clients。

有关此 Topic 的一些工具帮助，请查看 easyRSA 项目，该项目提供了大量的脚本来帮助完成这些步骤。

2.3.1. PEM 格式的 SSL 密钥和证书

原文引用：From 2.7.0 onwards, SSL key and trust stores can be configured for Kafka brokers and clients directly in the configuration in PEM format. This avoids the need to store separate files on the file system and benefits from password protection features of Kafka configuration. PEM may also be used as the store type for file-based key and trust stores in addition to JKS and PKCS12. To configure PEM key store directly in the broker or client configuration, private key in PEM format should be provided in ssl.keystore.key and the certificate chain in PEM format should be provided in ssl.keystore.certificate.chain. To configure trust store, trust certificates, e.g. public certificate of CA, should be provided in ssl.truststore.certificates. Since PEM is typically stored as multi-line base-64 strings, the configuration value can be included in Kafka configuration as multi-line strings with lines terminating in backslash ('') for line continuation.

从2.7.0开始，可以直接在 PEM 格式的配置中为 Kafka Broker 和客户端配置 SSL 密钥和信任存储。这避免了在文件系统上存储单独文件的需要，并得益于 Kafka 配置的密码保护功能。除了 JKS 和 PKCS12 之外，PEM 还可以用作基于文件的密钥和信任存储的存储类型。要直接在 Broker 或客户端配置中配置 PEM 密钥存储，应在 ssl.keystore.key 中提供 PEM 格式的私钥，并在 ssl.kkeystore.certificate.chain 中提供 PEM 格式的证书链。若要配置信任存储，应在ssl.truststore.certificates 中提供信任证书，例如 CA 的公共证书。由于 PEM 通常存储为多行 base-64 字符串，因此配置值可以作为多行字符串包含在 Kafka 配置中，其中的行以反斜杠（'\'）结尾，用于行延续。

原文引用：Store password configs ssl.keystore.password and ssl.truststore.password are not used for PEM. If private key is encrypted using a password, the key password must be provided in ssl.key.password. Private keys may be provided in unencrypted form without a password. In production deployments, configs should be encrypted or externalized using password protection feature in Kafka in this case. Note that the default SSL engine factory has limited capabilities for decryption of encrypted private keys when external tools like OpenSSL are used for encryption. Third party libraries like BouncyCastle may be integrated with a custom SslEngineFactory to support a wider range of encrypted private keys.

存储密码配置 ssl.keystore.password 和 ssl.truststore.password 不用于 PEM。如果使用密码对私钥进行加密，则必须在 ssl.key.password 中提供密钥密码。可以在没有密码的情况下以未加密的形式提供私钥。在生产部署中，应该使用 Kafka 中的密码保护功能对配置进行加密或外部化。请注意，当使用 OpenSSL 等外部工具进行加密时，默认的 SSL 引擎工厂对加密私钥的解密能力有限。BouncyCastle 等第三方库可以与自定义的 **SslEngineFactory **集成，以支持更广泛的加密私钥。