ElasticSearch-单节点迁移到集群S3方案

安装Minio

compose.yml

services:

  minio:
    container_name: minio
    image: minio/minio:RELEASE.2025-04-22T22-12-26Z
    ports:
#      - 9000:9000
      - 9001:9001
    volumes:
      - ./minio:/data
    environment:
      MINIO_ROOT_USER: root
      MINIO_ROOT_PASSWORD: QgittPhLFomW4vMD
#      MINIO_ROOT_PASSWORD: aFeCJD5zSprUDC3x
      TZ: Asia/Shanghai
    command: server /data --console-address ":9001"

# 注意，这里网络配置一定要正确
networks:
  default:
    external: true
    name: open-net

登陆控制台，创建账号

创建Buckets，并且设置为 public

ES安装S3插件

elasticsearch-plugin install repository-s3

温馨提示

不敏感信息只能添加到elasticsearch.yml中，敏感信息只能添加到keystore中。

也不能配置到docker启动参数中，因为此时S3插件还没有安装。

配置连接信息

s3.client.default:
  endpoint: "http://minio:9000"
  protocol: "http"
  path_style_access: true

配置敏感信息

echo "UbRsWWZHZ9plLSnA" | elasticsearch-keystore add s3.client.default.access_key
echo "DIYvD9IlEmKRVsAID4CRBPapfZZg3mJC" | elasticsearch-keystore add s3.client.default.secret_key

# 查看配置是否添加成功
elasticsearch-keystore list

# 参考显示如下
#bootstrap.password
#keystore.seed
#s3.client.default.access_key
#s3.client.default.secret_key

重启ES服务，确认插件是否安装成功。

GET /_cat/plugins?v

# 参考显示如下
#name          component     version
#elasticsearch repository-s3 7.17.6

PUT _snapshot/my_minio_backup
{
  "type": "s3",
  "settings": {
    "bucket": "elasticsearch-backups",
    "path_style_access": true,
    "compress": true,
    "max_snapshot_bytes_per_sec": "50mb",
    "max_restore_bytes_per_sec": "50mb"
  }
}

点击测试链接状态是否ok

数据备份

先测试几个小索引备份

POST /_snapshot/my_minio_backup/snapshot_wxmp_202312
{
  "indices": "gupaoedu-wxmp-msg-2023-12-*",
  "ignore_unavailable": true,
  "include_global_state": false
}

备份成功

数据恢复

在目标集群安装并配置S3插件

由于不在同一个网络，所以需要配置 endpoint。

s3.client.default:
  endpoint: "http://172.16.0.7:9000"
  protocol: "http"
  path_style_access: true

创建仓库

PUT _snapshot/my_minio_backup
{
  "type": "s3",
  "settings": {
    "bucket": "elasticsearch-backups",
    "endpoint": "http://172.16.0.7:9000",
    "path_style_access": true,
    "read_only": true,
    "max_snapshot_bytes_per_sec": "50mb",
    "max_restore_bytes_per_sec": "50mb"
  }
}

开始恢复

POST /_snapshot/my_minio_backup/snapshot_wxmp_202312/_restore
{
  "indices": "gupaoedu-wxmp-msg-2023-12-*",
  "ignore_unavailable": true,
  "include_global_state": false
}

数据恢复成功

数据顺利进入集群，分片和副本也处于健康状态。