【分布式文件存储】MinIO部署及实现文件上传下载

概述

MinIO | 高性能, Kubernetes 原生对象存储

MinIO是全球领先的对象存储先锋，目前在全世界有数百万的用户。

• 高性能 ，在标准硬件上，读/写速度上高达183GB/秒和171GB/秒，拥有更高的吞吐量和更低的延迟
• 可扩展性 ，为对象存储带来了简单的缩放模型，通过添加更多集群可以扩展空间
• 简单 ，极简主义是MinIO的指导性设计原则，即可在几分钟内安装和配置
• 与Amazon S3兼容 ，亚马逊云的 S3 API（接口协议）是在全球范围内达到共识的对象存储的协议，是全世界内大家都认可的标准
• 数据安全 ，使用纠删码来保护数据免受硬件故障和无声数据损坏

MinIO集群部署

基于docker-compose部署入门案例

准备docker-compose.yml

version: '3.7'

# 所有容器通用的设置和配置
x-minio-common: &minio-common
  image: minio/minio
  command: server --console-address ":9001" http://minio{1...4}/data
  expose:
    - "9000"
  # environment:
    # MINIO_ROOT_USER: minioadmin
    # MINIO_ROOT_PASSWORD: minioadmin
  healthcheck:
    test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
    interval: 30s
    timeout: 20s
    retries: 3

# 启动4个docker容器运行minio服务器实例
# 使用nginx反向代理9000端口，负载均衡, 你可以通过9001、9002、9003、9004端口访问它们的web console
services:
  minio1:
    <<: *minio-common
    hostname: minio1
    ports:
      - "9001:9001"
    volumes:
      - ./data/data1:/data

  minio2:
    <<: *minio-common
    hostname: minio2
    ports:
      - "9002:9001"
    volumes:
      - ./data/data2:/data

  minio3:
    <<: *minio-common
    hostname: minio3
    ports:
      - "9003:9001"
    volumes:
      - ./data/data3:/data

  minio4:
    <<: *minio-common
    hostname: minio4
    ports:
      - "9004:9001"
    volumes:
      - ./data/data4:/data

  nginx:
    image: nginx:1.19.2-alpine
    hostname: nginx
    volumes:
      - ./config/nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "9000:9000"
    depends_on:
      - minio1
      - minio2
      - minio3
      - minio4

在当前目录下，创建config，创建nginx.conf

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  4096;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    keepalive_timeout  65;

    # include /etc/nginx/conf.d/*.conf;

    upstream minio {
        server minio1:9000;
        server minio2:9000;
        server minio3:9000;
        server minio4:9000;
    }

    server {
        listen       9000;
        listen  [::]:9000;
        server_name  localhost;

        # To allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # To disable buffering
        proxy_buffering off;

        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            proxy_connect_timeout 300;
            # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            chunked_transfer_encoding off;

            proxy_pass http://minio;
        }
    }

}

测试启动

docker-compose up -d

访问任意节点

默认账户密码：minioadmin minioadmin

MinIO用户管理

需要创建一个用户，客户端用户可上传下载权限。

注意：长度最好是32位，否则后续调用API会报错

Buckets管理

MinIO uses buckets to organize objects. A bucket is similar to a folder or directory in a filesystem, where each bucket can hold an arbitrary number of objects.

MinIO使用bucket来组织对象。bucket类似于文件系统中的文件夹或目录，其中每个bucket可以容纳任意数量的对象。

创建Buckets

注意桶名的命名规范，否则后续调用api创建桶会报错

可以在控制台管理文件，上传下载预览删除等

Simple Storage Service（Amazon S3）存储桶命名要求 - AWS CloudTrail

MinIO客户端

官方接口API文档

Java Client API Reference — MinIO Object Storage for Linux

MinIO支持多种客户端，本文以java为例

引入依赖

<dependency>
    <groupId>io.minio</groupId>
    <artifactId>minio</artifactId>
    <version>8.5.3</version>
</dependency>

文件上传下载Demo

package pers.kw.minio.demo;

import com.alibaba.fastjson.JSON;
import io.minio.*;
import io.minio.errors.*;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;

import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.List;

public class MinIODemo {

    private static final String accessKey = "PPeCeQbHtFsEcGNxnlmSpbYasxELHoGy";
    private static final String secretKey = "GEMQDMUhQSHaIJCqmxperpESNYCJXvjJ";
    private static final String endpoint = "http://192.168.8.88:9000";

    public static void main(String[] args)
            throws ServerException, InsufficientDataException, ErrorResponseException,
            IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {
        String bucket = "test01";
        String objectName = "small125750JkEMk1625374670.jpg";
        downloadFile(bucket, objectName);

    }

    public static void downloadFile(String bucket, String objectName)
            throws ServerException, InsufficientDataException, ErrorResponseException, IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {
        MinioClient minioClient =
                MinioClient.builder()
                        .endpoint(endpoint)
                        .credentials(accessKey, secretKey)
                        .build();
        //todo 需要关闭流
        try (InputStream inputStream = minioClient.getObject(
                GetObjectArgs.builder()
                        .bucket(bucket)
                        .object(objectName)
                        .build())) {
            // Read data from stream
            byte[] bytes = IOUtils.toByteArray(inputStream);
            String fileName = "22.jpg";
            System.out.println("下载文件：" + bytes.length);
            FileUtils.writeByteArrayToFile(new File("C:\\Users\\pc\\Desktop\\" + fileName), bytes);
            System.out.println("文件下载成功");
            IOUtils.closeQuietly(inputStream);
        }
    }

    public static void uploadFile(String bucket) throws IOException, NoSuchAlgorithmException, InvalidKeyException {
        try {
            // Create a minioClient with the MinIO server playground, its access key and secret key.
            MinioClient minioClient =
                    MinioClient.builder()
                            .endpoint(endpoint)
                            .credentials(accessKey, secretKey)
                            .build();

            // Make 'asiatrip' bucket if not exist.
            boolean found =
                    minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucket).build());
            if (!found) {
                // Make a new bucket called bucket.
                minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucket).build());
            } else {
                System.out.println("Bucket " + bucket + " already exists.");
            }

            // Upload '/home/user/Photos/asiaphotos.zip' as object name 'asiaphotos-2015.zip' to bucket
            // 'asiatrip'.
            String fileName = "C:\\Users\\pc\\Desktop\\gateway-server.rar";
            String objectName = "gateway-server_2023058.rar";
            minioClient.uploadObject(
                    UploadObjectArgs.builder()
                            .bucket(bucket)
                            .object(objectName)
                            .filename(fileName)
                            .build());
            System.out.println(
                    fileName + " is successfully uploaded as"
                            + objectName + " to bucket " + bucket + ".");
        } catch (MinioException e) {
            System.out.println("Error occurred: " + e);
            System.out.println("HTTP trace: " + e.httpTrace());
        }
    }

}

调用API碰到的问题

问题一：

java.lang.NoSuchMethodError: okhttp3.RequestBody.create ( BLokhttp3/Medialype:)Lokhttp3/RequestBody;

解决: 引入如下依赖

<dependency>
    <groupId>com.squareup.okhttp3</groupId>
    <artifactId>okhttp</artifactId>
    <version>4.9.0</version>
</dependency>

问题二：

bucket name does not follow Amazon S3 standards.

命名不符合规则换个名字，最好在代码中增加正则校验

问题三：

RequestTimeTooSkewed

客户端时间与服务器时间相差过大导致

设置服务器时间

 ntpdate time.windows.com

问题4：

InvalidAccessKeyId

秘钥信息太短