Alex小新

模式

ElasticSearch-集群搭建 ​

系统环境设置 ​

由于ES对资源要求比较高，需要设置系统资源

修改文件句柄数 /etc/security/limits.conf

bash
* soft nofile 65536
* hard nofile 65536

通过 ulimit -a 查看当前系统资源限制

bash
ulimit -a

修改最大映射数量

bash
sysctl -w vm.max_map_count=262144

通过 docker-compose 快速搭建 ​

编辑 compose.yml 文件

yaml
services:
  
  es01:  # 这是您原有的单节点改造而来
    image: elastic/elasticsearch:7.17.6
    container_name: es01
    restart: always
    privileged: true
    user: root
    environment:
      - TZ=Asia/Shanghai
      - node.name=es01
      - cluster.name=chatsync-cluster  # 集群名称需要统一
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - "ES_JAVA_OPTS=-Xms2G -Xmx2G"
      - bootstrap.memory_lock=true
      - ELASTIC_PASSWORD=3CS37avDy6KXD74W
      - node.roles=master,data,ingest

      # 集群安全配置
      - xpack.security.enabled=true
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.keystore.path=elastic-certificates.p12
      - xpack.security.transport.ssl.truststore.path=elastic-certificates.p12
      - xpack.security.authc.api_key.enabled=true

      # 设置备份仓库权限
      - path.repo=/mnt/elasticsearch_backups  # 直接设置参数
    volumes:
      - ./certs/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
      - ./es01/data:/usr/share/elasticsearch/data
      - /mnt/elasticsearch_backups:/mnt/elasticsearch_backups
    ports:
      - 40002:9200
    ulimits:
      memlock:
        soft: -1
        hard: -1
  
  es02:
    image: elastic/elasticsearch:7.17.6
    container_name: es02
    restart: always
    privileged: true
    user: root
    environment:
      - TZ=Asia/Shanghai
      - node.name=es02
      - cluster.name=chatsync-cluster
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - "ES_JAVA_OPTS=-Xms2G -Xmx2G"
      - bootstrap.memory_lock=true
      - ELASTIC_PASSWORD=3CS37avDy6KXD74W
      - node.roles=master,data,ingest,voting_only
      # 集群安全配置
      - xpack.security.enabled=true
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.keystore.path=elastic-certificates.p12
      - xpack.security.transport.ssl.truststore.path=elastic-certificates.p12
      - xpack.security.authc.api_key.enabled=true
      # 设置备份仓库权限
      - "path.repo=/mnt/elasticsearch_backups"  # 直接设置参数
    volumes:
      - ./certs/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
      - ./es02/data:/usr/share/elasticsearch/data
      - /mnt/elasticsearch_backups:/mnt/elasticsearch_backups
    ulimits:
      memlock:
        soft: -1
        hard: -1

  es03:
    image: elastic/elasticsearch:7.17.6
    container_name: es03
    restart: always
    privileged: true
    user: root
    environment:
      - TZ=Asia/Shanghai
      - node.name=es03
      - cluster.name=chatsync-cluster
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - "ES_JAVA_OPTS=-Xms2G -Xmx2G"
      - bootstrap.memory_lock=true
      - ELASTIC_PASSWORD=3CS37avDy6KXD74W
      # - node.roles=data,ingest  # 这个节点不作为master候选
      - node.roles=master,data,ingest
      # 集群安全配置
      - xpack.security.enabled=true
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.keystore.path=elastic-certificates.p12
      - xpack.security.transport.ssl.truststore.path=elastic-certificates.p12
      - xpack.security.authc.api_key.enabled=true
      # 设置备份仓库权限
      - "path.repo=/mnt/elasticsearch_backups"  # 直接设置参数
    volumes:
      - ./certs/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
      - ./es03/data:/usr/share/elasticsearch/data
      - /mnt/elasticsearch_backups:/mnt/elasticsearch_backups
    ports:
      - 40002:9200
    ulimits:
      memlock:
        soft: -1
        hard: -1

  es-kibana:
    image: elastic/kibana:7.17.6
    container_name: es-kibana
    privileged: true
    environment:
      - TZ=Asia/Shanghai
      # - SERVER_BASEPATH=/csk
      - ELASTICSEARCH_HOSTS=http://es-nginx:9200
      - ELASTICSEARCH_USERNAME=elastic
      - ELASTICSEARCH_PASSWORD=7dA61ijrxcfBL5Jb
    ports:
      - 40001:5601

#  es-nginx:
#    container_name: es-nginx
#    # image: nginx:1.26.2-alpine # 有crontab  Alpine Linux v3.20 (不能自动恢复)
#    # image: nginx:1.28.0    （也不能自动恢复）
#    image: nginx:1.20.1
#    restart: always
#    ports:
#      - "40003:9200"
#      # - "443:443"
#    volumes:
#      - ./conf.d:/etc/nginx/conf.d
#      # - ./certs:/etc/nginx/certs


volumes:
  es01:
  es02:
  es03:

networks:
  default:
    external: true
    name: open-net

创建数据目录 ​

bash
#mkdir -p new-es01/data && chown -R 1000:1000 new-es01/data
mkdir -p es01/data && chown -R 1000:1000 es01/data
mkdir -p es02/data && chown -R 1000:1000 es02/data
mkdir -p es03/data && chown -R 1000:1000 es03/data

常见问题 ​

资源不够问题 ​

虚拟内存设置过小，需要设置vm.max_map_count

bash
es02  | ERROR: [1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch.
es02  | bootstrap check failure [1] of [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
es02  | ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/chatsync-cluster.log

解决方案

bash
sysctl -w vm.max_map_count=262144

Transport SSL 问题 ​

ES集群间通讯，开启 xpack.security.transport.ssl.enabled=true，需要配置证书

bash
ERROR: [1] bootstrap checks failed. 
bootstrap check failure [1] of [1]: 
Transport SSL must be enabled if security is enabled on a [basic] license. 
Please set [xpack.security.transport.ssl.enabled] to [true] 
or disable security by setting [xpack.security.enabled] to [false]

解决方案

进入es容器生成证书

bash
docker exec -it es01 bash
bin/elasticsearch-certutil cert -name transport-certs -out config/elastic-certificates.p12 -pass ""
# bin/elasticsearch-certutil cert -name transport-certs -out config/elastic-certificates.p12 -pass "" --days 60 -s 
exit
docker cp es01:/usr/share/elasticsearch/config/elastic-certificates.p12 .
yaml
es01:
  environment:
    - xpack.security.enabled=true
    - xpack.security.transport.ssl.enabled=true
    - xpack.security.transport.ssl.verification_mode=certificate
    - xpack.security.transport.ssl.keystore.path=elastic-certificates.p12
    - xpack.security.transport.ssl.truststore.path=elastic-certificates.p12
    - xpack.security.authc.api_key.enabled=true
  volumes:
    - ./certs/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12

常用命令 ​

查看节点统计信息 ​

bash
GET /_cluster/stats?human&pretty
json
{
  "_nodes" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "9dflySuCQOCWvfZqolNj3Q",
  "timestamp" : 1745483167660,
  "status" : "yellow",
  "indices" : {
    "count" : 1122,
    "shards" : {
      "total" : 1122,
      "primaries" : 1122,
      "replication" : 0.0,
      "index" : {
        "shards" : {
          "min" : 1,
          "max" : 1,
          "avg" : 1.0
        },
        "primaries" : {
          "min" : 1,
          "max" : 1,
          "avg" : 1.0
        },
        "replication" : {
          "min" : 0.0,
          "max" : 0.0,
          "avg" : 0.0
        }
      }
    },
    "docs" : {
      "count" : 166337061,
      "deleted" : 142932
    },
    "store" : {
      "size" : "94.2gb",
      "size_in_bytes" : 101187564452,
      "total_data_set_size" : "94.2gb",
      "total_data_set_size_in_bytes" : 101187564452,
      "reserved" : "0b",
      "reserved_in_bytes" : 0
    },
    "fielddata" : {
      "memory_size" : "45.8mb",
      "memory_size_in_bytes" : 48110832,
      "evictions" : 0
    },
    "query_cache" : {
      "memory_size" : "11.2mb",
      "memory_size_in_bytes" : 11792268,
      "total_count" : 32601951,
      "hit_count" : 1608513,
      "miss_count" : 30993438,
      "cache_size" : 624035,
      "cache_count" : 625950,
      "evictions" : 1915
    },
    "completion" : {
      "size" : "0b",
      "size_in_bytes" : 0
    },
    "segments" : {
      "count" : 10472,
      "memory" : "131.9mb",
      "memory_in_bytes" : 138372484,
      "terms_memory" : "97.3mb",
      "terms_memory_in_bytes" : 102116744,
      "stored_fields_memory" : "5.1mb",
      "stored_fields_memory_in_bytes" : 5391664,
      "term_vectors_memory" : "0b",
      "term_vectors_memory_in_bytes" : 0,
      "norms_memory" : "13.9mb",
      "norms_memory_in_bytes" : 14599808,
      "points_memory" : "0b",
      "points_memory_in_bytes" : 0,
      "doc_values_memory" : "15.5mb",
      "doc_values_memory_in_bytes" : 16264268,
      "index_writer_memory" : "1mb",
      "index_writer_memory_in_bytes" : 1137182,
      "version_map_memory" : "71.8kb",
      "version_map_memory_in_bytes" : 73579,
      "fixed_bit_set" : "7kb",
      "fixed_bit_set_memory_in_bytes" : 7224,
      "max_unsafe_auto_id_timestamp" : -1,
      "file_sizes" : { }
    },
    "mappings" : {
      "field_types" : [
        {
          "name" : "boolean",
          "count" : 602,
          "index_count" : 602,
          "script_count" : 0
        },
        {
          "name" : "constant_keyword",
          "count" : 6,
          "index_count" : 2,
          "script_count" : 0
        },
        {
          "name" : "date",
          "count" : 1145,
          "index_count" : 1109,
          "script_count" : 0
        },
        {
          "name" : "float",
          "count" : 580,
          "index_count" : 290,
          "script_count" : 0
        },
        {
          "name" : "ip",
          "count" : 2,
          "index_count" : 2,
          "script_count" : 0
        },
        {
          "name" : "keyword",
          "count" : 33129,
          "index_count" : 1111,
          "script_count" : 0
        },
        {
          "name" : "long",
          "count" : 11645,
          "index_count" : 1107,
          "script_count" : 0
        },
        {
          "name" : "nested",
          "count" : 4,
          "index_count" : 4,
          "script_count" : 0
        },
        {
          "name" : "object",
          "count" : 11232,
          "index_count" : 623,
          "script_count" : 0
        },
        {
          "name" : "text",
          "count" : 32913,
          "index_count" : 1111,
          "script_count" : 0
        },
        {
          "name" : "version",
          "count" : 4,
          "index_count" : 4,
          "script_count" : 0
        }
      ],
      "runtime_field_types" : [ ]
    },
    "analysis" : {
      "char_filter_types" : [ ],
      "tokenizer_types" : [ ],
      "filter_types" : [ ],
      "analyzer_types" : [ ],
      "built_in_char_filters" : [ ],
      "built_in_tokenizers" : [ ],
      "built_in_filters" : [ ],
      "built_in_analyzers" : [ ]
    },
    "versions" : [
      {
        "version" : "7.17.6",
        "index_count" : 1122,
        "primary_shard_count" : 1122,
        "total_primary_size" : "94.2gb",
        "total_primary_bytes" : 101187564452
      }
    ]
  },
  "nodes" : {
    "count" : {
      "total" : 1,
      "coordinating_only" : 0,
      "data" : 1,
      "data_cold" : 1,
      "data_content" : 1,
      "data_frozen" : 1,
      "data_hot" : 1,
      "data_warm" : 1,
      "ingest" : 1,
      "master" : 1,
      "ml" : 1,
      "remote_cluster_client" : 1,
      "transform" : 1,
      "voting_only" : 0
    },
    "versions" : [
      "7.17.6"
    ],
    "os" : {
      "available_processors" : 16,
      "allocated_processors" : 16,
      "names" : [
        {
          "name" : "Linux",
          "count" : 1
        }
      ],
      "pretty_names" : [
        {
          "pretty_name" : "Ubuntu 20.04.4 LTS",
          "count" : 1
        }
      ],
      "architectures" : [
        {
          "arch" : "amd64",
          "count" : 1
        }
      ],
      "mem" : {
        "total" : "30.6gb",
        "total_in_bytes" : 32870289408,
        "free" : "2.3gb",
        "free_in_bytes" : 2536771584,
        "used" : "28.2gb",
        "used_in_bytes" : 30333517824,
        "free_percent" : 8,
        "used_percent" : 92
      }
    },
    "process" : {
      "cpu" : {
        "percent" : 0
      },
      "open_file_descriptors" : {
        "min" : 11692,
        "max" : 11692,
        "avg" : 11692
      }
    },
    "jvm" : {
      "max_uptime" : "15.4h",
      "max_uptime_in_millis" : 55560470,
      "versions" : [
        {
          "version" : "18.0.2",
          "vm_name" : "OpenJDK 64-Bit Server VM",
          "vm_version" : "18.0.2+9-61",
          "vm_vendor" : "Oracle Corporation",
          "bundled_jdk" : true,
          "using_bundled_jdk" : true,
          "count" : 1
        }
      ],
      "mem" : {
        "heap_used" : "1.8gb",
        "heap_used_in_bytes" : 1958817888,
        "heap_max" : "4gb",
        "heap_max_in_bytes" : 4294967296
      },
      "threads" : 132
    },
    "fs" : {
      "total" : "787.3gb",
      "total_in_bytes" : 845443633152,
      "free" : "410.7gb",
      "free_in_bytes" : 441068494848,
      "available" : "378.6gb",
      "available_in_bytes" : 406585540608
    },
    "plugins" : [ ],
    "network_types" : {
      "transport_types" : {
        "security4" : 1
      },
      "http_types" : {
        "security4" : 1
      }
    },
    "discovery_types" : {
      "single-node" : 1
    },
    "packaging_types" : [
      {
        "flavor" : "default",
        "type" : "docker",
        "count" : 1
      }
    ],
    "ingest" : {
      "number_of_pipelines" : 10,
      "processor_stats" : {
        "conditional" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "convert" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "date" : {
          "count" : 89494,
          "failed" : 0,
          "current" : 0,
          "time" : "593ms",
          "time_in_millis" : 593
        },
        "geoip" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "grok" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "gsub" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "remove" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "rename" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "script" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "set" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "set_security_user" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        }
      }
    }
  }
}

查看索引明细 ​

bash
GET /_cat/indices?v&h=index,docs.count,store.size,pri.store.size&s=index
bash
index                           docs.count store.size pri.store.size
.apm-agent-configuration                 0       226b           226b
.apm-custom-link                         0       226b           226b
.async-search                            0       257b           257b
.fleet-enrollment-api-keys-7             2     11.7kb         11.7kb
.fleet-policies-7                        2     13.5kb         13.5kb
.geoip_databases                         9      7.4mb          7.4mb
.kibana_7.17.6_001                    2260      5.2mb          5.2mb
.kibana_task_manager_7.17.6_001         18     12.7mb         12.7mb
.security-7                             55    373.7kb        373.7kb
.tasks                                  22     37.5kb         37.5kb
gupaoedu-wxcp-msg-2025-04-17        531919    296.8mb        296.8mb
gupaoedu-wxcp-msg-2025-04-18        291948    223.5mb        223.5mb
gupaoedu-wxcp-msg-2025-04-19        232711      143mb          143mb
gupaoedu-wxcp-msg-2025-04-20         33580     27.5mb         27.5mb
gupaoedu-wxcp-msg-2025-04-21        311398    177.3mb        177.3mb
gupaoedu-wxcp-msg-2025-04-22        576987    365.3mb        365.3mb
gupaoedu-wxcp-msg-2025-04-23        513952    301.1mb        301.1mb
gupaoedu-wxcp-msg-2025-04-24         94685     69.5mb         69.5mb
gupaoedu-wxmp-msg-2025-04-18          6350      3.2mb          3.2mb
gupaoedu-wxmp-msg-2025-04-19          5402      2.8mb          2.8mb
gupaoedu-wxmp-msg-2025-04-20          5023      2.3mb          2.3mb
gupaoedu-wxmp-msg-2025-04-21          7899      3.7mb          3.7mb
gupaoedu-wxmp-msg-2025-04-22          7809      3.9mb          3.9mb
gupaoedu-wxmp-msg-2025-04-23          7777      3.7mb          3.7mb
gupaoedu-wxmp-msg-2025-04-24          3601      3.3mb          3.3mb

人生感悟

Copyright © 2019~2024 由vitepress强力驱动