bash test/kbcli/test_kbcli_0.9.sh --type 48 --version 0.9.5 --generate-output true --chaos-mesh true --drain-node true --random-namespace true --region eastus --cloud-provider aks CURRENT_TEST_DIR:test/kbcli source commons files source engines files source kubeblocks files `kubectl get namespace | grep ns-pwglh ` `kubectl create namespace ns-pwglh` namespace/ns-pwglh created create namespace ns-pwglh done download kbcli `gh release list --repo apecloud/kbcli --limit 100 | (grep "0.9" || true)` `curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v0.9.5-beta.8` Your system is linux_amd64 Installing kbcli ... Downloading ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 32.1M 100 32.1M 0 0 115M 0 --:--:-- --:--:-- --:--:-- 115M kbcli installed successfully. Kubernetes: v1.32.6 KubeBlocks: 0.9.5 kbcli: 0.9.5-beta.8 WARNING: version difference between kbcli (0.9.5-beta.8) and kubeblocks (0.9.5) Make sure your docker service is running and begin your journey with kbcli: kbcli playground init For more information on how to get started, please visit: https://kubeblocks.io download kbcli v0.9.5-beta.8 done Kubernetes: v1.32.6 KubeBlocks: 0.9.5 kbcli: 0.9.5-beta.8 WARNING: version difference between kbcli (0.9.5-beta.8) and kubeblocks (0.9.5) Kubernetes Env: v1.32.6 POD_RESOURCES: aks kb-default-sc found aks default-vsc found found default storage class: default kubeblocks version is:0.9.5 skip upgrade kubeblocks Error: no repositories to show helm repo add chaos-mesh https://charts.chaos-mesh.org "chaos-mesh" has been added to your repositories add helm chart repo chaos-mesh success chaos mesh already installed check cluster definition set component name:redis set component version set component version:redis-cluster set service versions:5.0.12,6.2.14,6.2.17,7.0.6,7.2.10,7.2.4,7.2.7,7.4.2,7.4.5,8.0.1,8.0.3 set service versions sorted:5.0.12,6.2.14,6.2.17,7.0.6,7.2.4,7.2.7,7.2.10,7.4.2,7.4.5,8.0.1,8.0.3 REPORT_COUNT 0:0 set replicas first:2,7.0.6|2,7.2.4|2,7.2.7|2,7.2.10|2,7.4.2|2,7.4.5|2,8.0.1|2,8.0.3 set replicas third:2,8.0.3 set replicas fourth:2,8.0.1 set minimum cmpv service version set minimum cmpv service version replicas:2,8.0.1 REPORT_COUNT:1 LIMIT_CPU:0.1 LIMIT_MEMORY:0.5 storage size: 1 No resources found in ns-pwglh namespace. termination_policy:WipeOut create 2 replica WipeOut redis cluster check cluster version check cluster definition check component definition set component definition by component version check cmpd by labels set component definition1: redis-cluster-8 by component version:redis-cluster apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: name: rediscl-vmwsrc namespace: ns-pwglh spec: terminationPolicy: WipeOut shardingSpecs: - name: shard shards: 3 template: name: redis componentDef: redis-cluster-8 serviceVersion: 8.0.1 replicas: 2 services: - name: redis-advertised serviceType: NodePort podService: true serviceAccountName: systemAccounts: - name: default passwordConfig: length: 10 numDigits: 5 numSymbols: 0 letterCase: MixedCases seed: rediscl-vmwsrc resources: limits: cpu: 100m memory: 0.5Gi requests: cpu: 100m memory: 0.5Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi `kubectl apply -f test_create_rediscl-vmwsrc.yaml` cluster.apps.kubeblocks.io/rediscl-vmwsrc created apply test_create_rediscl-vmwsrc.yaml Success `rm -rf test_create_rediscl-vmwsrc.yaml` check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Sep 01,2025 11:17 UTC+0800 cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 check pod status done check cluster role No resources found in ns-pwglh namespace. primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 rediscl-vmwsrc-shard-d9g-1;secondary: No resources found in ns-pwglh namespace. primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 rediscl-vmwsrc-shard-d9g-1;secondary: No resources found in ns-pwglh namespace. primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 rediscl-vmwsrc-shard-d9g-1;secondary: No resources found in ns-pwglh namespace. primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 rediscl-vmwsrc-shard-d9g-1;secondary: No resources found in ns-pwglh namespace. primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 rediscl-vmwsrc-shard-d9g-1;secondary: No resources found in ns-pwglh namespace. primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 rediscl-vmwsrc-shard-d9g-1;secondary: No resources found in ns-pwglh namespace. primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 rediscl-vmwsrc-shard-d9g-1;secondary: No resources found in ns-pwglh namespace. primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 rediscl-vmwsrc-shard-d9g-1;secondary: No resources found in ns-pwglh namespace. primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 rediscl-vmwsrc-shard-d9g-1;secondary: No resources found in ns-pwglh namespace. primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 rediscl-vmwsrc-shard-d9g-1;secondary: check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0 rediscl-vmwsrc-shard-d9g-1;secondary: rediscl-vmwsrc-shard-7wk-1 analyze cluster health check cluster connect `` ========== 节点: kb-post-provision-job-rediscl-vmwsrc-shard-4wn-f27rg ========== 无法识别分片编号，跳过该节点 ========== 节点: kb-post-provision-job-rediscl-vmwsrc-shard-7wk-n5wjd ========== 无法识别分片编号，跳过该节点 ========== 节点: kb-post-provision-job-rediscl-vmwsrc-shard-d9g-pmjdl ========== 无法识别分片编号，跳过该节点 ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756696815105 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756696815000 3 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 2 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756696816010 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:4 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:10 cluster_stats_messages_pong_sent:10 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:21 cluster_stats_messages_ping_received:10 cluster_stats_messages_pong_received:11 cluster_stats_messages_received:21 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249 master - 0 1756696817959 0 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 0 connected --- 集群状态信息 --- cluster_state:fail cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:5 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:0 cluster_stats_messages_ping_sent:6 cluster_stats_messages_pong_sent:1 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:8 cluster_stats_messages_ping_received:1 cluster_stats_messages_pong_received:7 cluster_stats_messages_received:8 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756696818000 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756696818000 3 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756696818844 2 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:4 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:11 cluster_stats_messages_pong_sent:16 cluster_stats_messages_sent:27 cluster_stats_messages_ping_received:13 cluster_stats_messages_pong_received:11 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:27 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756696818000 3 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756696819002 2 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756696819907 2 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756696818599 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:5 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:15 cluster_stats_messages_pong_sent:14 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:30 cluster_stats_messages_ping_received:14 cluster_stats_messages_pong_received:16 cluster_stats_messages_received:30 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756696820000 2 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 3 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232 master - 0 1756696821111 0 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756696820896 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756696820000 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756696819000 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:17 cluster_stats_messages_pong_sent:18 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:36 cluster_stats_messages_ping_received:17 cluster_stats_messages_pong_received:18 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:36 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475 master - 0 1756696821006 3 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201 slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756696821112 2 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756696821819 2 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756696821212 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723 master - 0 1756696821120 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:9 cluster_stats_messages_pong_sent:9 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:19 cluster_stats_messages_ping_received:9 cluster_stats_messages_pong_received:9 cluster_stats_messages_received:18 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 警告: 节点 rediscl-vmwsrc-shard-4wn-1 报告集群状态为 'fail'，期望值为 'ok' 警告: 节点 rediscl-vmwsrc-shard-4wn-1 报告已知节点数为 '5'，与第一个节点报告的 '4' 不一致 警告: 节点 rediscl-vmwsrc-shard-7wk-1 报告已知节点数为 '5'，与第一个节点报告的 '4' 不一致 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告已知节点数为 '6'，与第一个节点报告的 '4' 不一致 警告: 节点 rediscl-vmwsrc-shard-d9g-1 报告已知节点数为 '6'，与第一个节点报告的 '4' 不一致 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167 主节点 - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435 主节点 - 5461-10922 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971 主节点 - 0-5460 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 警告: 节点 rediscl-vmwsrc-shard-4wn-1 看到 2 个节点，而参考节点看到 4 个节点 警告: 节点 rediscl-vmwsrc-shard-4wn-1 缺少 3 个节点，它们在参考节点视图中存在 缺失的节点详情: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167 主节点 - 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971 主节点 - 0-5460 警告: 节点 rediscl-vmwsrc-shard-4wn-1 看到 1 个节点，它们在参考节点视图中不存在 额外的节点详情: ID IP地址 状态 ---------------------------------------- ------------- ---------------- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489 主节点 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 警告: 节点 rediscl-vmwsrc-shard-7wk-1 看到 5 个节点，而参考节点看到 4 个节点 警告: 节点 rediscl-vmwsrc-shard-7wk-1 看到 1 个节点，它们在参考节点视图中不存在 额外的节点详情: ID IP地址 状态 ---------------------------------------- ------------- ---------------- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489 从节点 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 警告: 节点 rediscl-vmwsrc-shard-d9g-0 看到 6 个节点，而参考节点看到 4 个节点 警告: 节点 rediscl-vmwsrc-shard-d9g-0 看到 2 个节点，它们在参考节点视图中不存在 额外的节点详情: ID IP地址 状态 ---------------------------------------- ------------- ---------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177 主节点 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489 从节点 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 警告: 节点 rediscl-vmwsrc-shard-d9g-1 看到 6 个节点，而参考节点看到 4 个节点 警告: 节点 rediscl-vmwsrc-shard-d9g-1 看到 2 个节点，它们在参考节点视图中不存在 额外的节点详情: ID IP地址 状态 ---------------------------------------- ------------- ---------------- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489 从节点 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177 从节点 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.7:32167) 负责 0 个槽位: 10923-16383 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.7:30435) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.7:30971) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 0 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 0 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 1 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 否 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 否 - 所有节点看到一致的拓扑结构: 否 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 不健康 ❌ 检测到以下问题: - 集群状态异常: 一个或多个节点报告集群状态不为OK - 集群视图不一致: 不同节点看到的集群大小或节点数量不同 - 拓扑结构不一致: 节点间对集群成员或角色的认知不一致 建议措施: 4. 使用 CLUSTER MEET 命令修复节点间的连接 5. 确保所有节点的配置(如cluster-announce-ip)正确 health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697038513 3 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697038916 3 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697037507 2 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697039520 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 2 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697038000 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:417 cluster_stats_messages_pong_sent:416 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:834 cluster_stats_messages_ping_received:415 cluster_stats_messages_pong_received:418 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:834 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697039680 2 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697040385 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 2 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697040687 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697040000 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697039000 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:402 cluster_stats_messages_pong_sent:404 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:807 cluster_stats_messages_ping_received:404 cluster_stats_messages_pong_received:403 cluster_stats_messages_received:807 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697040157 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697040661 3 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697041667 2 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697040000 3 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697040057 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:402 cluster_stats_messages_pong_sent:413 cluster_stats_messages_sent:815 cluster_stats_messages_ping_received:410 cluster_stats_messages_pong_received:402 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:815 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697041000 3 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697042260 2 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697042000 2 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697042562 3 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697042479 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:426 cluster_stats_messages_pong_sent:415 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:842 cluster_stats_messages_ping_received:415 cluster_stats_messages_pong_received:427 cluster_stats_messages_received:842 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697043010 2 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 3 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697043513 3 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697044016 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697043000 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697043000 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:408 cluster_stats_messages_pong_sent:413 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:822 cluster_stats_messages_ping_received:412 cluster_stats_messages_pong_received:409 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:822 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697045000 3 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697044000 2 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697045592 2 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697045000 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697045289 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:412 cluster_stats_messages_pong_sent:411 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:824 cluster_stats_messages_ping_received:411 cluster_stats_messages_pong_received:413 cluster_stats_messages_received:824 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167 主节点 - 10923-16383 (共 0 个槽位) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435 主节点 - 5461-10922 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971 主节点 - 0-5460 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.7:32167) 负责 0 个槽位: 10923-16383 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.7:30435) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.7:30971) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl get secrets -l app.kubernetes.io/instance=rediscl-vmwsrc` set secret: rediscl-vmwsrc-shard-default `kubectl get secrets rediscl-vmwsrc-shard-default -o jsonpath="***.data.username***"` `kubectl get secrets rediscl-vmwsrc-shard-default -o jsonpath="***.data.password***"` `kubectl get secrets rediscl-vmwsrc-shard-default -o jsonpath="***.data.port***"` DB_USERNAME:default;DB_PASSWORD:550AWr4Z7M;DB_PORT:6379;DB_DATABASE: check pod rediscl-vmwsrc-shard-4wn-0 container_name redis-cluster exist password 550AWr4Z7M check pod rediscl-vmwsrc-shard-4wn-0 container_name lorry exist password 550AWr4Z7M check pod rediscl-vmwsrc-shard-4wn-0 container_name config-manager exist password 550AWr4Z7M No container logs contain secret password. describe cluster `kbcli cluster describe rediscl-vmwsrc --namespace ns-pwglh ` Name: rediscl-vmwsrc Created Time: Sep 01,2025 11:17 UTC+0800 NAMESPACE CLUSTER-DEFINITION VERSION STATUS TERMINATION-POLICY ns-pwglh Running WipeOut Endpoints: COMPONENT MODE INTERNAL EXTERNAL Topology: COMPONENT INSTANCE ROLE STATUS AZ NODE CREATED-TIME shard-4wn rediscl-vmwsrc-shard-4wn-0 primary Running 0 aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 shard-4wn rediscl-vmwsrc-shard-4wn-1 secondary Running 0 aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 shard-7wk rediscl-vmwsrc-shard-7wk-0 primary Running 0 aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 shard-7wk rediscl-vmwsrc-shard-7wk-1 secondary Running 0 aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 shard-d9g rediscl-vmwsrc-shard-d9g-0 primary Running 0 aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 shard-d9g rediscl-vmwsrc-shard-d9g-1 secondary Running 0 aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 Resources Allocation: COMPONENT DEDICATED CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS Images: COMPONENT TYPE IMAGE Data Protection: BACKUP-REPO AUTO-BACKUP BACKUP-SCHEDULE BACKUP-METHOD BACKUP-RETENTION RECOVERABLE-TIME Show cluster events: kbcli cluster list-events -n ns-pwglh rediscl-vmwsrc get shard component name set shard component name:shard-4wn `kbcli cluster label rediscl-vmwsrc app.kubernetes.io/instance- --namespace ns-pwglh ` label "app.kubernetes.io/instance" not found. `kbcli cluster label rediscl-vmwsrc app.kubernetes.io/instance=rediscl-vmwsrc --namespace ns-pwglh ` `kbcli cluster label rediscl-vmwsrc --list --namespace ns-pwglh ` NAME NAMESPACE LABELS rediscl-vmwsrc ns-pwglh app.kubernetes.io/instance=rediscl-vmwsrc label cluster app.kubernetes.io/instance=rediscl-vmwsrc Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=rediscl-vmwsrc --namespace ns-pwglh ` `kbcli cluster label rediscl-vmwsrc --list --namespace ns-pwglh ` NAME NAMESPACE LABELS rediscl-vmwsrc ns-pwglh app.kubernetes.io/instance=rediscl-vmwsrc case.name=kbcli.test1 label cluster case.name=kbcli.test1 Success `kbcli cluster label rediscl-vmwsrc case.name=kbcli.test2 --overwrite --namespace ns-pwglh ` `kbcli cluster label rediscl-vmwsrc --list --namespace ns-pwglh ` NAME NAMESPACE LABELS rediscl-vmwsrc ns-pwglh app.kubernetes.io/instance=rediscl-vmwsrc case.name=kbcli.test2 label cluster case.name=kbcli.test2 Success `kbcli cluster label rediscl-vmwsrc case.name- --namespace ns-pwglh ` `kbcli cluster label rediscl-vmwsrc --list --namespace ns-pwglh ` NAME NAMESPACE LABELS rediscl-vmwsrc ns-pwglh app.kubernetes.io/instance=rediscl-vmwsrc delete cluster label case.name Success cluster connect kbcli unsupported engine type: shard-4wn cluster hscale check cluster status before ops check cluster status done cluster_status:Running No resources found in rediscl-vmwsrc namespace. `kbcli cluster hscale rediscl-vmwsrc --auto-approve --force=true --components shard --replicas 3 --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-horizontalscaling-gb76x created successfully, you can view the progress: kbcli cluster describe-ops rediscl-vmwsrc-horizontalscaling-gb76x -n ns-pwglh check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-horizontalscaling-gb76x ns-pwglh HorizontalScaling rediscl-vmwsrc shard Pending -/- Sep 01,2025 11:25 UTC+0800 check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Updating Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 rediscl-vmwsrc-shard-4wn-2 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:25 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 rediscl-vmwsrc-shard-7wk-2 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:25 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 rediscl-vmwsrc-shard-d9g-2 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:25 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-4wn-2 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-7wk-2 rediscl-vmwsrc-shard-d9g-1 rediscl-vmwsrc-shard-d9g-2 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- e938984504edc267597efeee3f5cb5659bda5123 10.224.0.10:30861@30929,rediscl-vmwsrc-shard-7wk-2.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697201536 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697201536 3 connected 709aa63fd8f0d3f22001d5a37bd5571a37f4d04e 10.224.0.10:30556@31889,rediscl-vmwsrc-shard-4wn-2.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697200529 2 connected 243d677a4d1d977edc7e005d3c0e6b59da48f9b6 10.224.0.10:30594@31138,rediscl-vmwsrc-shard-d9g-2.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697200000 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697201000 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697201000 2 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697200000 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697200529 3 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 2 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:709 cluster_stats_messages_pong_sent:725 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1435 cluster_stats_messages_ping_received:720 cluster_stats_messages_pong_received:710 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:1435 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- e938984504edc267597efeee3f5cb5659bda5123 10.224.0.10:30861@30929,rediscl-vmwsrc-shard-7wk-2.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697201569 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697202576 1 connected 0-5460 709aa63fd8f0d3f22001d5a37bd5571a37f4d04e 10.224.0.10:30556@31889,rediscl-vmwsrc-shard-4wn-2.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697202476 2 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697201569 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697201468 3 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697201569 2 connected 5461-10922 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 2 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697200966 3 connected 10923-16383 243d677a4d1d977edc7e005d3c0e6b59da48f9b6 10.224.0.10:30594@31138,rediscl-vmwsrc-shard-d9g-2.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697202000 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:698 cluster_stats_messages_pong_sent:722 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1421 cluster_stats_messages_ping_received:722 cluster_stats_messages_pong_received:699 cluster_stats_messages_received:1421 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-2 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 2 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 709aa63fd8f0d3f22001d5a37bd5571a37f4d04e 10.224.0.10:30556@31889,rediscl-vmwsrc-shard-4wn-2.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 2 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697204000 1 connected 0-5460 e938984504edc267597efeee3f5cb5659bda5123 10.224.0.10:30861@30929,rediscl-vmwsrc-shard-7wk-2.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697203582 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697204790 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697203582 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697203281 3 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697203080 2 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697203784 2 connected 5461-10922 243d677a4d1d977edc7e005d3c0e6b59da48f9b6 10.224.0.10:30594@31138,rediscl-vmwsrc-shard-d9g-2.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697204588 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:65 cluster_stats_messages_pong_sent:54 cluster_stats_messages_meet_sent:4 cluster_stats_messages_sent:123 cluster_stats_messages_ping_received:54 cluster_stats_messages_pong_received:69 cluster_stats_messages_received:123 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697205328 1 connected e938984504edc267597efeee3f5cb5659bda5123 10.224.0.10:30861@30929,rediscl-vmwsrc-shard-7wk-2.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697204825 1 connected 243d677a4d1d977edc7e005d3c0e6b59da48f9b6 10.224.0.10:30594@31138,rediscl-vmwsrc-shard-d9g-2.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697205000 3 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697204321 2 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697204019 2 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697205831 3 connected 709aa63fd8f0d3f22001d5a37bd5571a37f4d04e 10.224.0.10:30556@31889,rediscl-vmwsrc-shard-4wn-2.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697204522 2 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697205530 3 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:715 cluster_stats_messages_pong_sent:714 cluster_stats_messages_sent:1429 cluster_stats_messages_ping_received:706 cluster_stats_messages_pong_received:714 cluster_stats_messages_meet_received:8 cluster_stats_messages_received:1428 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697205599 3 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697205000 2 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697206000 2 connected 5461-10922 709aa63fd8f0d3f22001d5a37bd5571a37f4d04e 10.224.0.10:30556@31889,rediscl-vmwsrc-shard-4wn-2.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697206606 2 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697206000 1 connected 0-5460 243d677a4d1d977edc7e005d3c0e6b59da48f9b6 10.224.0.10:30594@31138,rediscl-vmwsrc-shard-d9g-2.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697206505 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected e938984504edc267597efeee3f5cb5659bda5123 10.224.0.10:30861@30929,rediscl-vmwsrc-shard-7wk-2.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697204794 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697206505 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:741 cluster_stats_messages_pong_sent:726 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1468 cluster_stats_messages_ping_received:726 cluster_stats_messages_pong_received:742 cluster_stats_messages_received:1468 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-2 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 2 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697207000 2 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697206000 2 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697207000 3 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697206000 1 connected 0-5460 243d677a4d1d977edc7e005d3c0e6b59da48f9b6 10.224.0.10:30594@31138,rediscl-vmwsrc-shard-d9g-2.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697206709 3 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697207724 3 connected 10923-16383 e938984504edc267597efeee3f5cb5659bda5123 10.224.0.10:30861@30929,rediscl-vmwsrc-shard-7wk-2.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected 709aa63fd8f0d3f22001d5a37bd5571a37f4d04e 10.224.0.10:30556@31889,rediscl-vmwsrc-shard-4wn-2.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697207000 2 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697206105 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:69 cluster_stats_messages_pong_sent:57 cluster_stats_messages_meet_sent:4 cluster_stats_messages_sent:130 cluster_stats_messages_ping_received:57 cluster_stats_messages_pong_received:73 cluster_stats_messages_received:130 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 3 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697208591 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697207079 3 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697207000 2 connected e938984504edc267597efeee3f5cb5659bda5123 10.224.0.10:30861@30929,rediscl-vmwsrc-shard-7wk-2.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697208692 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697208000 2 connected 5461-10922 709aa63fd8f0d3f22001d5a37bd5571a37f4d04e 10.224.0.10:30556@31889,rediscl-vmwsrc-shard-4wn-2.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697207684 2 connected 243d677a4d1d977edc7e005d3c0e6b59da48f9b6 10.224.0.10:30594@31138,rediscl-vmwsrc-shard-d9g-2.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697207583 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697208591 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:713 cluster_stats_messages_pong_sent:732 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1446 cluster_stats_messages_ping_received:727 cluster_stats_messages_pong_received:714 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:1446 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697209515 3 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697209000 2 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 3 connected e938984504edc267597efeee3f5cb5659bda5123 10.224.0.10:30861@30929,rediscl-vmwsrc-shard-7wk-2.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697209515 1 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697208509 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697209515 1 connected 0-5460 243d677a4d1d977edc7e005d3c0e6b59da48f9b6 10.224.0.10:30594@31138,rediscl-vmwsrc-shard-d9g-2.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697209000 3 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697209515 2 connected 709aa63fd8f0d3f22001d5a37bd5571a37f4d04e 10.224.0.10:30556@31889,rediscl-vmwsrc-shard-4wn-2.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697209000 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:720 cluster_stats_messages_pong_sent:739 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1460 cluster_stats_messages_ping_received:739 cluster_stats_messages_pong_received:721 cluster_stats_messages_received:1460 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-2 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 2 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697209501 3 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697209000 3 connected 10923-16383 e938984504edc267597efeee3f5cb5659bda5123 10.224.0.10:30861@30929,rediscl-vmwsrc-shard-7wk-2.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697210205 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697209703 2 connected 243d677a4d1d977edc7e005d3c0e6b59da48f9b6 10.224.0.10:30594@31138,rediscl-vmwsrc-shard-d9g-2.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 3 connected 709aa63fd8f0d3f22001d5a37bd5571a37f4d04e 10.224.0.10:30556@31889,rediscl-vmwsrc-shard-4wn-2.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697209199 2 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697209000 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697211212 2 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697211000 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:82 cluster_stats_messages_pong_sent:65 cluster_stats_messages_meet_sent:5 cluster_stats_messages_sent:152 cluster_stats_messages_ping_received:65 cluster_stats_messages_pong_received:87 cluster_stats_messages_received:152 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- e938984504edc267597efeee3f5cb5659bda5123 10.224.0.10:30861 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b 709aa63fd8f0d3f22001d5a37bd5571a37f4d04e 10.224.0.10:30556 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 243d677a4d1d977edc7e005d3c0e6b59da48f9b6 10.224.0.10:30594 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971 主节点 - 0-5460 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167 主节点 - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435 主节点 - 5461-10922 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-4wn-2 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-2 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-2 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.7:30971) 负责 0 个槽位: 0-5460 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.7:32167) 负责 0 个槽位: 10923-16383 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.7:30435) 负责 0 个槽位: 5461-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 2 个从节点 ✅ 分片 7wk: 1 个主节点, 2 个从节点 ✅ 分片 d9g: 1 个主节点, 2 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 6 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success No resources found in rediscl-vmwsrc namespace. check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-horizontalscaling-gb76x ns-pwglh HorizontalScaling rediscl-vmwsrc shard Succeed 3/3 Sep 01,2025 11:25 UTC+0800 check ops status done ops_status:rediscl-vmwsrc-horizontalscaling-gb76x ns-pwglh HorizontalScaling rediscl-vmwsrc shard Succeed 3/3 Sep 01,2025 11:25 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests rediscl-vmwsrc-horizontalscaling-gb76x --namespace ns-pwglh ` opsrequest.apps.kubeblocks.io/rediscl-vmwsrc-horizontalscaling-gb76x patched `kbcli cluster delete-ops --name rediscl-vmwsrc-horizontalscaling-gb76x --force --auto-approve --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-horizontalscaling-gb76x deleted cluster hscale check cluster status before ops check cluster status done cluster_status:Running No resources found in rediscl-vmwsrc namespace. `kbcli cluster hscale rediscl-vmwsrc --auto-approve --force=true --components shard --replicas 2 --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-horizontalscaling-zq559 created successfully, you can view the progress: kbcli cluster describe-ops rediscl-vmwsrc-horizontalscaling-zq559 -n ns-pwglh check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-horizontalscaling-zq559 ns-pwglh HorizontalScaling rediscl-vmwsrc shard Pending -/- Sep 01,2025 11:27 UTC+0800 check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Running Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697285000 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697285512 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697284507 2 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697285000 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697284000 3 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 2 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:918 cluster_stats_messages_pong_sent:946 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1865 cluster_stats_messages_ping_received:941 cluster_stats_messages_pong_received:919 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:1865 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697285000 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697285288 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697285087 3 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697286000 2 connected 5461-10922 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 2 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697286294 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:910 cluster_stats_messages_pong_sent:939 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1850 cluster_stats_messages_ping_received:939 cluster_stats_messages_pong_received:911 cluster_stats_messages_received:1850 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697287405 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697287103 2 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697286000 2 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697287000 3 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697286399 3 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:924 cluster_stats_messages_pong_sent:921 cluster_stats_messages_sent:1845 cluster_stats_messages_ping_received:913 cluster_stats_messages_pong_received:923 cluster_stats_messages_meet_received:8 cluster_stats_messages_received:1844 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697288812 3 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697288308 2 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697288000 2 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697289315 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697288000 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:956 cluster_stats_messages_pong_sent:925 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1882 cluster_stats_messages_ping_received:925 cluster_stats_messages_pong_received:957 cluster_stats_messages_received:1882 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 3 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697289523 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697288013 3 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697289523 2 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697288516 2 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697289523 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:912 cluster_stats_messages_pong_sent:933 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1846 cluster_stats_messages_ping_received:928 cluster_stats_messages_pong_received:913 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:1846 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697290000 3 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697290688 2 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697291000 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697291000 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756697290000 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:916 cluster_stats_messages_pong_sent:944 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1861 cluster_stats_messages_ping_received:944 cluster_stats_messages_pong_received:917 cluster_stats_messages_received:1861 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971 主节点 - 0-5460 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167 主节点 - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435 主节点 - 5461-10922 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.7:30971) 负责 0 个槽位: 0-5460 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.7:32167) 负责 0 个槽位: 10923-16383 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.7:30435) 负责 0 个槽位: 5461-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success No resources found in rediscl-vmwsrc namespace. check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-horizontalscaling-zq559 ns-pwglh HorizontalScaling rediscl-vmwsrc shard Succeed 3/3 Sep 01,2025 11:27 UTC+0800 check ops status done ops_status:rediscl-vmwsrc-horizontalscaling-zq559 ns-pwglh HorizontalScaling rediscl-vmwsrc shard Succeed 3/3 Sep 01,2025 11:27 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests rediscl-vmwsrc-horizontalscaling-zq559 --namespace ns-pwglh ` opsrequest.apps.kubeblocks.io/rediscl-vmwsrc-horizontalscaling-zq559 patched `kbcli cluster delete-ops --name rediscl-vmwsrc-horizontalscaling-zq559 --force --auto-approve --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-horizontalscaling-zq559 deleted test failover fullcpu check cluster status before cluster-failover-fullcpu check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-rediscl-vmwsrc" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-rediscl-vmwsrc" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-fullcpu-rediscl-vmwsrc namespace: ns-pwglh spec: selector: namespaces: - ns-pwglh labelSelectors: apps.kubeblocks.io/pod-name: rediscl-vmwsrc-shard-4wn-0 mode: all stressors: cpu: workers: 100 load: 100 duration: 2m `kubectl apply -f test-chaos-mesh-fullcpu-rediscl-vmwsrc.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-fullcpu-rediscl-vmwsrc created apply test-chaos-mesh-fullcpu-rediscl-vmwsrc.yaml Success `rm -rf test-chaos-mesh-fullcpu-rediscl-vmwsrc.yaml` fullcpu chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Updating Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697488000 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697487557 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697488683 4 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697487657 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697488280 3 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:4 cluster_stats_messages_ping_sent:1087 cluster_stats_messages_pong_sent:1091 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:2179 cluster_stats_messages_ping_received:1086 cluster_stats_messages_pong_received:1088 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:2179 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697489000 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697487875 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697489000 3 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697489587 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 4 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697489889 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:4 cluster_stats_messages_ping_sent:1245 cluster_stats_messages_pong_sent:1286 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:2543 cluster_stats_messages_ping_received:1281 cluster_stats_messages_pong_received:1245 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:2528 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697490000 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697490896 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697489589 4 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697489000 3 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697490594 3 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:1 cluster_stats_messages_ping_sent:1255 cluster_stats_messages_pong_sent:1274 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:2531 cluster_stats_messages_ping_received:1266 cluster_stats_messages_pong_received:1254 cluster_stats_messages_meet_received:8 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:2530 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697490595 3 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697490595 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697491000 4 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697492306 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697491803 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:1 cluster_stats_messages_ping_sent:1292 cluster_stats_messages_pong_sent:1264 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:2558 cluster_stats_messages_ping_received:1264 cluster_stats_messages_pong_received:1293 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:2559 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 3 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697492105 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697493582 3 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697493280 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697492709 4 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697492508 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:3 cluster_stats_messages_ping_sent:1259 cluster_stats_messages_pong_sent:1275 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:2536 cluster_stats_messages_ping_received:1270 cluster_stats_messages_pong_received:1259 cluster_stats_messages_meet_received:5 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:2536 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697493180 3 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697493681 4 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697494183 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697492535 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697493000 4 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:3 cluster_stats_messages_ping_sent:1248 cluster_stats_messages_pong_sent:1264 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:2514 cluster_stats_messages_ping_received:1264 cluster_stats_messages_pong_received:1248 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:2514 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489 主节点 - 5461-10922 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971 主节点 - 0-5460 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167 主节点 - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.7:32489) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.7:30971) 负责 0 个槽位: 0-5460 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.7:32167) 负责 0 个槽位: 10923-16383 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-rediscl-vmwsrc" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-rediscl-vmwsrc" not found check failover pod name failover pod name:rediscl-vmwsrc-shard-4wn-1 failover fullcpu Failure test failover timeoffset check cluster status before cluster-failover-timeoffset check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-rediscl-vmwsrc" not found Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-rediscl-vmwsrc" not found apiVersion: chaos-mesh.org/v1alpha1 kind: TimeChaos metadata: name: test-chaos-mesh-timeoffset-rediscl-vmwsrc namespace: ns-pwglh spec: selector: namespaces: - ns-pwglh labelSelectors: apps.kubeblocks.io/pod-name: rediscl-vmwsrc-shard-4wn-1 mode: all timeOffset: '-10m' clockIds: - CLOCK_REALTIME duration: 2m `kubectl apply -f test-chaos-mesh-timeoffset-rediscl-vmwsrc.yaml` timechaos.chaos-mesh.org/test-chaos-mesh-timeoffset-rediscl-vmwsrc created apply test-chaos-mesh-timeoffset-rediscl-vmwsrc.yaml Success `rm -rf test-chaos-mesh-timeoffset-rediscl-vmwsrc.yaml` timeoffset chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Running Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:18 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697686000 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697686955 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697685546 4 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697685949 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697686000 3 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:4 cluster_stats_messages_ping_sent:1447 cluster_stats_messages_pong_sent:1459 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:2907 cluster_stats_messages_ping_received:1454 cluster_stats_messages_pong_received:1448 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:2907 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697687714 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697687714 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697687000 3 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697687000 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 4 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697688118 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:4 cluster_stats_messages_ping_sent:1593 cluster_stats_messages_pong_sent:1639 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:3244 cluster_stats_messages_ping_received:1634 cluster_stats_messages_pong_received:1593 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:3229 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697688000 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697688000 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697689583 4 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697688878 3 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697689583 3 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:1 cluster_stats_messages_ping_sent:1618 cluster_stats_messages_pong_sent:1642 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:3262 cluster_stats_messages_ping_received:1634 cluster_stats_messages_pong_received:1617 cluster_stats_messages_meet_received:8 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:3261 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697689000 3 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697689000 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697689354 4 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697690058 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697690360 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:1 cluster_stats_messages_ping_sent:1653 cluster_stats_messages_pong_sent:1642 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:3297 cluster_stats_messages_ping_received:1642 cluster_stats_messages_pong_received:1654 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:3298 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== ��测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 3 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697690796 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756697690594 3 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697691803 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697691601 4 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697690594 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:3 cluster_stats_messages_ping_sent:1621 cluster_stats_messages_pong_sent:1654 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:3277 cluster_stats_messages_ping_received:1649 cluster_stats_messages_pong_received:1621 cluster_stats_messages_meet_received:5 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:3277 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697692708 3 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697691701 4 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 3 connected b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697692000 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697692506 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697692506 4 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:3 cluster_stats_messages_ping_sent:1609 cluster_stats_messages_pong_sent:1582 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:3193 cluster_stats_messages_ping_received:1582 cluster_stats_messages_pong_received:1609 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:3193 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.7:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.7:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.7:32489 主节点 - 5461-10922 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.7:30971 主节点 - 0-5460 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.7:32167 主节点 - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.7:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.7:32489) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.7:30971) 负责 0 个槽位: 0-5460 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.7:32167) 负责 0 个槽位: 10923-16383 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-rediscl-vmwsrc" force deleted Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-rediscl-vmwsrc" not found check failover pod name failover pod name:rediscl-vmwsrc-shard-4wn-1 failover timeoffset Success cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart rediscl-vmwsrc --auto-approve --force=true --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-restart-j47h2 created successfully, you can view the progress: kbcli cluster describe-ops rediscl-vmwsrc-restart-j47h2 -n ns-pwglh check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-restart-j47h2 ns-pwglh Restart rediscl-vmwsrc shard Creating -/- Sep 01,2025 11:36 UTC+0800 ops_status:rediscl-vmwsrc-restart-j47h2 ns-pwglh Restart rediscl-vmwsrc shard Creating -/- Sep 01,2025 11:36 UTC+0800 check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Updating Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:36 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:37 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:37 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:36 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:38 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:36 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-1;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697975485 4 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697974478 7 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697975585 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697975000 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756697974000 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:4 cluster_stats_messages_ping_sent:255 cluster_stats_messages_pong_sent:253 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:524 cluster_stats_messages_ping_received:253 cluster_stats_messages_pong_received:252 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:512 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697977000 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697977571 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697977067 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756697976000 7 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697976563 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 4 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:4 cluster_stats_messages_ping_sent:180 cluster_stats_messages_pong_sent:167 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:353 cluster_stats_messages_ping_received:166 cluster_stats_messages_pong_received:179 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:348 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697978510 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697978000 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697978000 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756697978510 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697977102 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:1 cluster_stats_messages_ping_sent:179 cluster_stats_messages_pong_sent:176 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:363 cluster_stats_messages_ping_received:176 cluster_stats_messages_pong_received:180 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:358 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697979522 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697979000 4 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756697978000 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697979000 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697979925 4 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:1 cluster_stats_messages_ping_sent:256 cluster_stats_messages_pong_sent:257 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:14 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:533 cluster_stats_messages_ping_received:257 cluster_stats_messages_pong_received:253 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:516 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756697980064 7 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697981071 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697979059 4 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697980568 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697979562 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:57 cluster_stats_messages_pong_sent:36 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:94 cluster_stats_messages_ping_received:36 cluster_stats_messages_pong_received:58 cluster_stats_messages_received:94 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756697981434 4 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756697982038 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756697981535 4 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756697980529 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756697982440 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:161 cluster_stats_messages_pong_sent:135 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:4 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:306 cluster_stats_messages_ping_received:129 cluster_stats_messages_pong_received:161 cluster_stats_messages_meet_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:293 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点 - 5461-10922 (共 0 个槽位) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177 主节点 - 10923-16383 (共 0 个槽位) b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 从节点 f49b9ec6a80833cb30167969e9f74558c9513f47 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 5461-10922 节点 f49b9ec6a80833cb30167969e9f74558c9513f47 (10.224.0.10:32177) 负责 0 个槽位: 10923-16383 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-restart-j47h2 ns-pwglh Restart rediscl-vmwsrc shard Succeed 6/6 Sep 01,2025 11:36 UTC+0800 check ops status done ops_status:rediscl-vmwsrc-restart-j47h2 ns-pwglh Restart rediscl-vmwsrc shard Succeed 6/6 Sep 01,2025 11:36 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests rediscl-vmwsrc-restart-j47h2 --namespace ns-pwglh ` opsrequest.apps.kubeblocks.io/rediscl-vmwsrc-restart-j47h2 patched `kbcli cluster delete-ops --name rediscl-vmwsrc-restart-j47h2 --force --auto-approve --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-restart-j47h2 deleted test failover networkdelay check cluster status before cluster-failover-networkdelay check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-rediscl-vmwsrc" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-rediscl-vmwsrc" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkdelay-rediscl-vmwsrc namespace: ns-pwglh spec: selector: namespaces: - ns-pwglh labelSelectors: apps.kubeblocks.io/pod-name: rediscl-vmwsrc-shard-4wn-1 mode: all action: delay delay: latency: 2000ms correlation: '100' jitter: 0ms direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkdelay-rediscl-vmwsrc.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkdelay-rediscl-vmwsrc created apply test-chaos-mesh-networkdelay-rediscl-vmwsrc.yaml Success `rm -rf test-chaos-mesh-networkdelay-rediscl-vmwsrc.yaml` networkdelay chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Updating Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:36 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:37 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:37 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:36 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:38 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:36 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-1;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698179000 4 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698179005 7 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698178000 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698179508 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698180113 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:4 cluster_stats_messages_ping_sent:611 cluster_stats_messages_pong_sent:602 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1229 cluster_stats_messages_ping_received:602 cluster_stats_messages_pong_received:608 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:1217 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698180000 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698180000 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698180963 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698180000 7 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698179554 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 4 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:4 cluster_stats_messages_ping_sent:478 cluster_stats_messages_pong_sent:447 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:931 cluster_stats_messages_ping_received:446 cluster_stats_messages_pong_received:477 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:926 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698182197 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698181000 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698181000 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698182701 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698181191 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:1 cluster_stats_messages_ping_sent:545 cluster_stats_messages_pong_sent:543 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:1096 cluster_stats_messages_ping_received:543 cluster_stats_messages_pong_received:546 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:1091 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698182000 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698181505 4 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698182511 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698182511 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698183518 4 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:1 cluster_stats_messages_ping_sent:610 cluster_stats_messages_pong_sent:614 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:14 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1244 cluster_stats_messages_ping_received:614 cluster_stats_messages_pong_received:607 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:1227 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698184555 7 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698184053 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698183000 4 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698184000 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698183550 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:405 cluster_stats_messages_pong_sent:408 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:814 cluster_stats_messages_ping_received:408 cluster_stats_messages_pong_received:406 cluster_stats_messages_received:814 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698185870 4 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698185568 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698185568 4 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698184000 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698185000 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:520 cluster_stats_messages_pong_sent:496 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:4 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1026 cluster_stats_messages_ping_received:490 cluster_stats_messages_pong_received:519 cluster_stats_messages_meet_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:1012 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点 - 5461-10922 (共 0 个槽位) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177 主节点 - 10923-16383 (共 0 个槽位) b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 从节点 f49b9ec6a80833cb30167969e9f74558c9513f47 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 5461-10922 节点 f49b9ec6a80833cb30167969e9f74558c9513f47 (10.224.0.10:32177) 负责 0 个槽位: 10923-16383 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-rediscl-vmwsrc" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-rediscl-vmwsrc" not found check failover pod name failover pod name:rediscl-vmwsrc-shard-4wn-1 failover networkdelay Success 6 test failover podkill check cluster status before cluster-failover-podkill check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podkill-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podkill-rediscl-vmwsrc" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podkill-rediscl-vmwsrc" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podkill-rediscl-vmwsrc namespace: ns-pwglh spec: selector: namespaces: - ns-pwglh labelSelectors: apps.kubeblocks.io/pod-name: rediscl-vmwsrc-shard-4wn-1 mode: all action: pod-kill `kubectl apply -f test-chaos-mesh-podkill-rediscl-vmwsrc.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podkill-rediscl-vmwsrc created apply test-chaos-mesh-podkill-rediscl-vmwsrc.yaml Success `rm -rf test-chaos-mesh-podkill-rediscl-vmwsrc.yaml` check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Updating Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:36 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:44 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:37 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:36 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:38 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:36 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-1;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698283000 4 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698283295 7 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698282000 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698282289 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698282591 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:4 cluster_stats_messages_ping_sent:790 cluster_stats_messages_pong_sent:797 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1603 cluster_stats_messages_ping_received:797 cluster_stats_messages_pong_received:786 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:1590 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698284000 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698284000 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698284602 7 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698284000 4 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698284501 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:4 cluster_stats_messages_ping_sent:47 cluster_stats_messages_pong_sent:28 cluster_stats_messages_meet_sent:2 cluster_stats_messages_sent:77 cluster_stats_messages_ping_received:28 cluster_stats_messages_pong_received:48 cluster_stats_messages_received:76 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698286066 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698285000 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698284556 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698284000 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698285059 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:1 cluster_stats_messages_ping_sent:722 cluster_stats_messages_pong_sent:728 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:1458 cluster_stats_messages_ping_received:727 cluster_stats_messages_pong_received:722 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:1452 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698287260 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698286554 4 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698286252 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698286000 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698286554 4 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:1 cluster_stats_messages_ping_sent:793 cluster_stats_messages_pong_sent:791 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:14 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1604 cluster_stats_messages_ping_received:791 cluster_stats_messages_pong_received:789 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:1586 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698288377 7 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698287000 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698286566 4 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698287371 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698287000 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:587 cluster_stats_messages_pong_sent:586 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1174 cluster_stats_messages_ping_received:586 cluster_stats_messages_pong_received:587 cluster_stats_messages_received:1173 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698288668 4 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698288000 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698289675 4 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698288000 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698289000 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:702 cluster_stats_messages_pong_sent:678 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:4 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1390 cluster_stats_messages_ping_received:671 cluster_stats_messages_pong_received:700 cluster_stats_messages_meet_received:2 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:1375 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点 - 5461-10922 (共 0 个槽位) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177 主节点 - 10923-16383 (共 0 个槽位) b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 从节点 f49b9ec6a80833cb30167969e9f74558c9513f47 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 5461-10922 节点 f49b9ec6a80833cb30167969e9f74558c9513f47 (10.224.0.10:32177) 负责 0 个槽位: 10923-16383 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podkill-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podkill-rediscl-vmwsrc" force deleted podchaos.chaos-mesh.org/test-chaos-mesh-podkill-rediscl-vmwsrc patched check failover pod name failover pod name:rediscl-vmwsrc-shard-4wn-1 failover podkill Success `kubectl get secrets -l app.kubernetes.io/instance=rediscl-vmwsrc` set secret: rediscl-vmwsrc-shard-4wn-account-default `kubectl get secrets rediscl-vmwsrc-shard-4wn-account-default -o jsonpath="***.data.username***"` `kubectl get secrets rediscl-vmwsrc-shard-4wn-account-default -o jsonpath="***.data.password***"` `kubectl get secrets rediscl-vmwsrc-shard-4wn-account-default -o jsonpath="***.data.port***"` DB_USERNAME:default;DB_PASSWORD:550AWr4Z7M;DB_PORT:6379;DB_DATABASE: apiVersion: v1 kind: Pod metadata: name: benchtest-rediscl-vmwsrc namespace: ns-pwglh spec: containers: - name: test-benchmark imagePullPolicy: IfNotPresent image: docker.io/apecloud/redis-benchmark:latest args: - "-h" - "rediscl-vmwsrc-shard-4wn-redis-advertised-0.ns-pwglh.svc.cluster.local" - "-p" - "6379" - "-a" - "550AWr4Z7M" - "-n" - "500" - "-c" - "4" - "--cluster" - "-q" restartPolicy: Never `kubectl apply -f benchtest-rediscl-vmwsrc.yaml` pod/benchtest-rediscl-vmwsrc created apply benchtest-rediscl-vmwsrc.yaml Success check pod status pod_status:NAME READY STATUS RESTARTS AGE benchtest-rediscl-vmwsrc 0/1 ContainerCreating 0 0s pod_status:NAME READY STATUS RESTARTS AGE benchtest-rediscl-vmwsrc 1/1 Running 0 4s pod_status:NAME READY STATUS RESTARTS AGE benchtest-rediscl-vmwsrc 1/1 Running 0 9s check pod benchtest-rediscl-vmwsrc status done pod_status:NAME READY STATUS RESTARTS AGE benchtest-rediscl-vmwsrc 0/1 Completed 0 14s `rm -rf benchtest-rediscl-vmwsrc.yaml` `kubectl logs benchtest-rediscl-vmwsrc --tail 30 --namespace ns-pwglh ` Cluster has 3 master nodes: Master 0: d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 Master 1: f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177 Master 2: f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 PING_INLINE: rps=0.0 (overall: 3000.0) avg_msec=0.073 (overall: 0.089) PING_INLINE: 1992.03 requests per second, p50=0.135 msec PING_MBULK: rps=51.6 (overall: 13000.0) avg_msec=0.075 (overall: 0.077) PING_MBULK: 1992.03 requests per second, p50=0.119 msec SET: rps=23.9 (overall: inf) avg_msec=0.073 (overall: 0.083) SET: 1992.03 requests per second, p50=0.143 msec GET: rps=31.6 (overall: 8000.0) avg_msec=0.079 (overall: 0.072) GET: 1992.03 requests per second, p50=0.119 msec INCR: rps=12.0 (overall: inf) avg_msec=0.093 (overall: 0.143) INCR: 1992.03 requests per second, p50=0.127 msec LPUSH: rps=39.7 (overall: 10000.0) avg_msec=0.076 (overall: 0.084) LPUSH: 1984.13 requests per second, p50=0.127 msec RPUSH: rps=39.5 (overall: 10000.0) avg_msec=0.066 (overall: 0.073) RPUSH: 1992.03 requests per second, p50=0.143 msec LPOP: rps=27.9 (overall: inf) avg_msec=0.062 (overall: 0.075) LPOP: 2000.00 requests per second, p50=0.127 msec RPOP: rps=35.7 (overall: 9000.0) avg_msec=0.068 (overall: 0.080) RPOP: 1992.03 requests per second, p50=0.127 msec SADD: rps=51.6 (overall: 13000.0) avg_msec=0.078 (overall: 0.076) SADD: 1992.03 requests per second, p50=0.111 msec HSET: rps=19.9 (overall: 5000.0) avg_msec=0.077 (overall: 0.074) HSET: 1992.03 requests per second, p50=0.135 msec SPOP: rps=27.8 (overall: 7000.0) avg_msec=0.080 (overall: 0.088) SPOP: 1992.03 requests per second, p50=0.119 msec ZADD: rps=35.7 (overall: 9000.0) avg_msec=0.102 (overall: 0.083) ZADD: 1992.03 requests per second, p50=0.119 msec ZPOPMIN: rps=47.6 (overall: 12000.0) avg_msec=0.063 (overall: 0.063) ZPOPMIN: 1992.03 requests per second, p50=0.127 msec LPUSH (needed to benchmark LRANGE): rps=27.9 (overall: 7000.0) avg_msec=0.070 (overall: 0.070) LPUSH (needed to benchmark LRANGE): 1992.03 requests per second, p50=0.127 msec LRANGE_100 (first 100 elements): rps=23.8 (overall: 6000.0) avg_msec=0.075 (overall: 0.073) LRANGE_100 (first 100 elements): 1984.13 requests per second, p50=0.127 msec LRANGE_300 (first 300 elements): rps=23.8 (overall: 6000.0) avg_msec=0.088 (overall: 0.102) LRANGE_300 (first 300 elements): 1992.03 requests per second, p50=0.135 msec LRANGE_500 (first 500 elements): rps=19.8 (overall: 5000.0) avg_msec=0.079 (overall: 0.086) LRANGE_500 (first 500 elements): 1992.03 requests per second, p50=0.127 msec LRANGE_600 (first 600 elements): rps=15.9 (overall: 4000.0) avg_msec=0.086 (overall: 0.080) LRANGE_600 (first 600 elements): 1992.03 requests per second, p50=0.135 msec MSET (10 keys): rps=27.8 (overall: 7000.0) avg_msec=0.106 (overall: 0.092) MSET (10 keys): 1992.03 requests per second, p50=0.135 msec XADD: rps=15.9 (overall: 4000.0) avg_msec=0.085 (overall: 0.117) XADD: 1992.03 requests per second, p50=0.135 msec `kubectl delete pod benchtest-rediscl-vmwsrc --force --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "benchtest-rediscl-vmwsrc" force deleted test delete pod all `kubectl get pod -n ns-pwglh | grep rediscl-vmwsrc-shard | awk '***print $1***' | xargs kubectl delete pod -n ns-pwglh --force ` test delete pod all times: 1 Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "rediscl-vmwsrc-shard-4wn-0" force deleted pod "rediscl-vmwsrc-shard-4wn-1" force deleted pod "rediscl-vmwsrc-shard-7wk-0" force deleted pod "rediscl-vmwsrc-shard-7wk-1" force deleted pod "rediscl-vmwsrc-shard-d9g-0" force deleted pod "rediscl-vmwsrc-shard-d9g-1" force deleted check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Running Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Init:0/2 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:47 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Init:0/2 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:47 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Init:0/2 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:47 UTC+0800 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 check pod status done check cluster role No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. primary: ;secondary: No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. primary: ;secondary: No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. primary: ;secondary: No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. primary: ;secondary: No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. primary: ;secondary: check cluster role done primary: rediscl-vmwsrc-shard-7wk-0;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-d9g-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698501253 1756698498733 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698498733 1756698498733 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698500751 1756698498733 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave,fail? f49b9ec6a80833cb30167969e9f74558c9513f47 1756698499747 1756698498733 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave,fail? f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 1756698501253 1756698498733 1 connected --- 集群状态信息 --- cluster_state:fail cluster_slots_assigned:16384 cluster_slots_ok:0 cluster_slots_pfail:16384 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:4 cluster_stats_messages_ping_sent:4 cluster_stats_messages_sent:4 cluster_stats_messages_received:0 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249 slave,fail? d4512197faed320ad44935144bd1e67f8e4e1925 1756698498770 1756698497738 4 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698500277 1756698497739 7 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave,fail? f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 1756698500277 1756698497738 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.10:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698508358 7 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698499774 1756698497738 4 connected 5461-10922 --- 集群状态信息 --- cluster_state:fail cluster_slots_assigned:16384 cluster_slots_ok:5461 cluster_slots_pfail:10923 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:1 cluster_stats_messages_ping_sent:7 cluster_stats_messages_pong_sent:2 cluster_stats_messages_sent:9 cluster_stats_messages_ping_received:2 cluster_stats_messages_pong_received:2 cluster_stats_messages_received:4 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.10:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698508324 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249 slave,fail? d4512197faed320ad44935144bd1e67f8e4e1925 1756698499844 1756698497832 4 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698500347 1756698497832 7 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698497832 1756698497832 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave,fail? f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 1756698500347 1756698497832 1 connected --- 集群状态信息 --- cluster_state:fail cluster_slots_assigned:16384 cluster_slots_ok:5461 cluster_slots_pfail:10923 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:6 cluster_stats_messages_pong_sent:4 cluster_stats_messages_sent:10 cluster_stats_messages_ping_received:4 cluster_stats_messages_pong_received:2 cluster_stats_messages_received:6 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 警告: 节点 rediscl-vmwsrc-shard-4wn-0 报告集群状态为 'fail'，期望值为 'ok' 警告: 节点 rediscl-vmwsrc-shard-4wn-0 报告正常槽位数为 '0'，期望与分配的槽位数相同 警告: 节点 rediscl-vmwsrc-shard-4wn-0 报告 16384 个疑似故障槽位和 0 个故障槽位 警告: 节点 rediscl-vmwsrc-shard-4wn-0 报告集群中存在故障或疑似故障节点 警告: 节点 rediscl-vmwsrc-shard-7wk-0 报告集群状态为 'fail'，期望值为 'ok' 警告: 节点 rediscl-vmwsrc-shard-7wk-0 报告正常槽位数为 '5461'，期望与分配的槽位数相同 警告: 节点 rediscl-vmwsrc-shard-7wk-0 报告 10923 个疑似故障槽位和 0 个故障槽位 警告: 节点 rediscl-vmwsrc-shard-7wk-0 报告集群中存在故障或疑似故障节点 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告集群状态为 'fail'，期望值为 'ok' 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告正常槽位数为 '5461'，期望与分配的槽位数相同 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告 10923 个疑似故障槽位和 0 个故障槽位 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告集群中存在故障或疑似故障节点 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点(故障) - 0-5460 (共 0 个槽位) d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点(故障) - 5461-10922 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177 主节点(故障) - 10923-16383 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 从节点(故障) f49b9ec6a80833cb30167969e9f74558c9513f47 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点(故障) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 检测到 5 个故障或疑似故障节点: ID IP地址 状态 所属分片 报告时间 ---------------------------------------- ------------- ---------------- ----------- ------------------- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点(故障) 7wk 5460 (epoch) d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点(故障) 未知 10922 (epoch) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.10:32177 主节点(故障) 未知 16383 (epoch) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 从节点(故障) d9g 未知 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点(故障) 未知 未知 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 警告: 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 的故障状态在不同视图中不一致 参考节点视图: f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698501253 1756698498733 1 connected 0-5460 (fail) 当前节点视图: f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 (正常) 警告: 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 的故障状态在不同视图中不一致 参考节点视图: fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected (正常) 当前节点视图: fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249 slave,fail? d4512197faed320ad44935144bd1e67f8e4e1925 1756698498770 1756698497738 4 connected (fail) 警告: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b 的故障状态在不同视图中不一致 参考节点视图: 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave,fail? f49b9ec6a80833cb30167969e9f74558c9513f47 1756698499747 1756698498733 7 connected (fail) 当前节点视图: 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.10:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698508358 7 connected (正常) 警告: 发现 3 个节点的角色、主从关系或故障状态不一致 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 警告: 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 的故障状态在不同视图中不一致 参考节点视图: f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698501253 1756698498733 1 connected 0-5460 (fail) 当前节点视图: f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698508324 1 connected 0-5460 (正常) 警告: 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 的故障状态在不同视图中不一致 参考节点视图: fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected (正常) 当前节点视图: fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249 slave,fail? d4512197faed320ad44935144bd1e67f8e4e1925 1756698499844 1756698497832 4 connected (fail) 警告: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b 的故障状态在不同视图中不一致 参考节点视图: 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave,fail? f49b9ec6a80833cb30167969e9f74558c9513f47 1756698499747 1756698498733 7 connected (fail) 当前节点视图: 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.10:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected (正常) 警告: 发现 3 个节点的角色、主从关系或故障状态不一致 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 5461-10922 节点 f49b9ec6a80833cb30167969e9f74558c9513f47 (10.224.0.10:32177) 负责 0 个槽位: 10923-16383 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 0 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 0 个从节点, 1 个故障节点 ❌ 分片 d9g: 0 个主节点, 1 个从节点, 1 个故障节点 ❌ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 5 个故障或疑似故障节点 - 集群状态统一显示为OK: 否 - 所有槽位正确分配: 否 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 否 集群健康状态: 不健康 ❌ 检测到以下问题: - 集群状态异常: 一个或多个节点报告集群状态不为OK - 槽位分配问题: 槽位数量不正确或存在故障槽位 - 存在故障节点: 检测到一个或多个节点被标记为故障或疑似故障 建议措施: 1. 检查故障节点的网络连接和运行状态 2. 对于故障从节点，考虑重启或替换它们 3. 对于故障主节点，确认是否已自动进行故障转移 health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使�� Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698729634 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698728627 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698729000 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.10:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698728526 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698729000 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:9 cluster_my_epoch:4 cluster_stats_messages_ping_sent:371 cluster_stats_messages_pong_sent:372 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:9 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:758 cluster_stats_messages_ping_received:372 cluster_stats_messages_pong_received:368 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:748 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698730047 4 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.10:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698730449 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698729946 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698729000 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698729442 7 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 4 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:9 cluster_my_epoch:4 cluster_stats_messages_ping_sent:335 cluster_stats_messages_pong_sent:330 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:675 cluster_stats_messages_ping_received:328 cluster_stats_messages_pong_received:334 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:671 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698731000 4 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698731152 7 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698732059 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.10:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698730145 7 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698731000 4 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:9 cluster_my_epoch:1 cluster_stats_messages_ping_sent:367 cluster_stats_messages_pong_sent:392 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:768 cluster_stats_messages_ping_received:391 cluster_stats_messages_pong_received:364 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:765 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698731090 7 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698733103 4 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.10:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698732000 7 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698732600 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698732096 4 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:9 cluster_my_epoch:1 cluster_stats_messages_ping_sent:343 cluster_stats_messages_pong_sent:311 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:660 cluster_stats_messages_ping_received:311 cluster_stats_messages_pong_received:343 cluster_stats_messages_received:654 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.10:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698733000 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698734000 4 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756698733740 7 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698733000 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698733000 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:9 cluster_my_epoch:7 cluster_stats_messages_ping_sent:372 cluster_stats_messages_pong_sent:376 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:754 cluster_stats_messages_ping_received:376 cluster_stats_messages_pong_received:369 cluster_stats_messages_fail_received:8 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:755 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698734000 4 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756698734000 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.10:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698734578 7 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756698735585 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756698735584 4 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:9 cluster_my_epoch:7 cluster_stats_messages_ping_sent:335 cluster_stats_messages_pong_sent:334 cluster_stats_messages_fail_sent:9 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:679 cluster_stats_messages_ping_received:332 cluster_stats_messages_pong_received:333 cluster_stats_messages_meet_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:668 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971 主节点 - 0-5460 (共 0 个槽位) d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点 - 5461-10922 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 主节点 - 10923-16383 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.10:32167 从节点 f49b9ec6a80833cb30167969e9f74558c9513f47 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.10:30971) 负责 0 个槽位: 0-5460 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 5461-10922 节点 f49b9ec6a80833cb30167969e9f74558c9513f47 (10.224.0.9:32177) 负责 0 个槽位: 10923-16383 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success test delete pod all times: 2 Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "rediscl-vmwsrc-shard-4wn-0" force deleted pod "rediscl-vmwsrc-shard-4wn-1" force deleted pod "rediscl-vmwsrc-shard-7wk-0" force deleted pod "rediscl-vmwsrc-shard-7wk-1" force deleted pod "rediscl-vmwsrc-shard-d9g-0" force deleted pod "rediscl-vmwsrc-shard-d9g-1" force deleted check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Running Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Init:0/2 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:53 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Init:0/2 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:53 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Init:0/2 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:53 UTC+0800 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 Init:0/2 Init:0/2 pod_status:Init:0/2 pod_status:Init:0/2 check pod status done check cluster role No resources found in ns-pwglh namespace. primary: ;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-d9g-0 No resources found in ns-pwglh namespace. primary: ;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-d9g-0 No resources found in ns-pwglh namespace. primary: ;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-d9g-0 No resources found in ns-pwglh namespace. primary: ;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-d9g-0 No resources found in ns-pwglh namespace. primary: ;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-d9g-0 No resources found in ns-pwglh namespace. primary: ;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-d9g-0 No resources found in ns-pwglh namespace. primary: ;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-d9g-0 check cluster role done primary: rediscl-vmwsrc-shard-7wk-0;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-d9g-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698828086 1756698825570 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave,fail? f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 1756698828086 1756698825570 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698848890 7 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698827583 1756698825570 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698826578 1756698825570 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected --- 集群状态信息 --- cluster_state:fail cluster_slots_assigned:16384 cluster_slots_ok:0 cluster_slots_pfail:16384 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:9 cluster_my_epoch:4 cluster_stats_messages_ping_sent:14 cluster_stats_messages_pong_sent:21 cluster_stats_messages_sent:35 cluster_stats_messages_ping_received:21 cluster_stats_messages_pong_received:9 cluster_stats_messages_received:30 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 无法从环境变量中获取Redis端口，使用默认端口6379 --- 集群节点信息 --- 执行命令失败 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave,fail? f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 1756698841111 1756698838568 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698839602 1756698838568 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.10:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756698850856 7 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698849796 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698841111 1756698838568 4 disconnected 5461-10922 --- 集群状态信息 --- cluster_state:fail cluster_slots_assigned:16384 cluster_slots_ok:5461 cluster_slots_pfail:10923 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:9 cluster_my_epoch:1 cluster_stats_messages_ping_sent:18 cluster_stats_messages_sent:18 cluster_stats_messages_pong_received:15 cluster_stats_messages_received:15 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave,fail? f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 1756698828090 1756698825570 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698827587 1756698825570 7 disconnected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698828090 1756698825570 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698828090 1756698825570 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756698852365 4 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected --- 集群状态信息 --- cluster_state:fail cluster_slots_assigned:16384 cluster_slots_ok:0 cluster_slots_pfail:16384 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:9 cluster_my_epoch:7 cluster_stats_messages_ping_sent:20 cluster_stats_messages_pong_sent:17 cluster_stats_messages_sent:37 cluster_stats_messages_ping_received:17 cluster_stats_messages_pong_received:15 cluster_stats_messages_received:32 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 无法从环境变量中获取Redis端口，使用默认端口6379 --- 集群节点信息 --- 执行命令失败 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 警告: 节点 rediscl-vmwsrc-shard-4wn-0 报告集群状态为 'fail'，期望值为 'ok' 警告: 节点 rediscl-vmwsrc-shard-4wn-0 报告正常槽位数为 '0'，期望与分配的槽位数相同 警告: 节点 rediscl-vmwsrc-shard-4wn-0 报告 16384 个疑似故障槽位和 0 个故障槽位 警告: 节点 rediscl-vmwsrc-shard-4wn-0 报告集群中存在故障或疑似故障节点 警告: 节点 rediscl-vmwsrc-shard-7wk-0 报告集群状态为 'fail'，期望值为 'ok' 警告: 节点 rediscl-vmwsrc-shard-7wk-0 报告正常槽位数为 '5461'，期望与分配的槽位数相同 警告: 节点 rediscl-vmwsrc-shard-7wk-0 报告 10923 个疑似故障槽位和 0 个故障槽位 警告: 节点 rediscl-vmwsrc-shard-7wk-0 报告集群中存在故障或疑似故障节点 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告集群状态为 'fail'，期望值为 'ok' 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告正常槽位数为 '0'，期望与分配的槽位数相同 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告 16384 个疑似故障槽位和 0 个故障槽位 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告集群中存在故障或疑似故障节点 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点(故障) - 5461-10922 (共 0 个槽位) b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点(故障) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 从节点 f49b9ec6a80833cb30167969e9f74558c9513f47 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点(故障) - 0-5460 (共 0 个槽位) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 主节点(故障) - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 检测到 4 个故障或疑似故障节点: ID IP地址 状态 所属分片 报告时间 ---------------------------------------- ------------- ---------------- ----------- ------------------- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点(故障) 未知 10922 (epoch) b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点(故障) 未知 未知 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点(故障) 7wk 5460 (epoch) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 主节点(故障) 未知 16383 (epoch) 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 警告: 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 的故障状态在不同视图中不一致 参考节点视图: f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master,fail? - 1756698827583 1756698825570 1 connected 0-5460 (fail) 当前节点视图: f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 (正常) 警告: 发现 1 个节点的角色、主从关系或故障状态不一致 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 节点 f49b9ec6a80833cb30167969e9f74558c9513f47 (10.224.0.9:32177) 负责 0 个槽位: 10923-16383 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 0 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 0 个从节点, 1 个故障节点 ❌ 分片 d9g: 0 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 4 个故障或疑似故障节点 - 集群状态统一显示为OK: 否 - 所有槽位正确分配: 否 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 否 集群健康状态: 不健康 ❌ 检测到以下问题: - 集群状态异常: 一个或多个节点报告集群状态不为OK - 槽位分配问题: 槽位数量不正确或存在故障槽位 - 存在故障节点: 检测到一个或多个节点被标记为故障或疑似故障 建议措施: 1. 检查故障节点的网络连接和运行状态 2. 对于故障从节点，考虑重启或替换它们 3. 对于故障主节点，确认是否已自动进行故障转移 health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699072366 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699072000 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699071661 7 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699072668 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699071560 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:4 cluster_stats_messages_ping_sent:393 cluster_stats_messages_pong_sent:427 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:841 cluster_stats_messages_ping_received:427 cluster_stats_messages_pong_received:389 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:823 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699074001 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699074000 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699073000 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699073000 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699074505 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 4 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:4 cluster_stats_messages_ping_sent:362 cluster_stats_messages_pong_sent:371 cluster_stats_messages_fail_sent:9 cluster_stats_messages_sent:742 cluster_stats_messages_ping_received:369 cluster_stats_messages_pong_received:360 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:735 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699075071 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699076078 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699075575 7 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699074567 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699074064 4 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:1 cluster_stats_messages_ping_sent:402 cluster_stats_messages_pong_sent:395 cluster_stats_messages_meet_sent:2 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:801 cluster_stats_messages_ping_received:394 cluster_stats_messages_pong_received:401 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:805 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699076211 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699077217 4 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699076110 7 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699076513 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699076000 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:1 cluster_stats_messages_ping_sent:364 cluster_stats_messages_pong_sent:338 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:703 cluster_stats_messages_ping_received:338 cluster_stats_messages_pong_received:365 cluster_stats_messages_received:703 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699077367 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699078373 7 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699077000 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699077000 4 connected 5461-10922 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699078000 4 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:7 cluster_stats_messages_ping_sent:404 cluster_stats_messages_pong_sent:409 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:824 cluster_stats_messages_ping_received:409 cluster_stats_messages_pong_received:400 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:818 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699077956 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699078559 4 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699078559 7 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699078961 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699078000 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:7 cluster_stats_messages_ping_sent:380 cluster_stats_messages_pong_sent:357 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:747 cluster_stats_messages_ping_received:355 cluster_stats_messages_pong_received:378 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:738 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点 - 5461-10922 (共 0 个槽位) b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 从节点 f49b9ec6a80833cb30167969e9f74558c9513f47 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 主节点 - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 节点 f49b9ec6a80833cb30167969e9f74558c9513f47 (10.224.0.9:32177) 负责 0 个槽位: 10923-16383 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success test delete pod all times: 3 Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "rediscl-vmwsrc-shard-4wn-0" force deleted pod "rediscl-vmwsrc-shard-4wn-1" force deleted pod "rediscl-vmwsrc-shard-7wk-0" force deleted pod "rediscl-vmwsrc-shard-7wk-1" force deleted pod "rediscl-vmwsrc-shard-d9g-0" force deleted pod "rediscl-vmwsrc-shard-d9g-1" force deleted check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Running Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Init:0/2 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Init:0/2 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Init:0/2 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 pod_status:Init:0/2 Init:0/2 Init:0/2 check pod status done check cluster role No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. primary: ;secondary: No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. primary: ;secondary: No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. primary: ;secondary: No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. primary: ;secondary: No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. primary: ;secondary: check cluster role done primary: rediscl-vmwsrc-shard-7wk-0;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-d9g-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master,fail? - 1756699148430 1756699146499 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master,fail? - 1756699149033 1756699146499 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave,fail? f49b9ec6a80833cb30167969e9f74558c9513f47 1756699149033 1756699146499 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave,fail? f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 1756699147426 1756699146499 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master,fail? - 1756699149033 1756699146499 4 connected 5461-10922 --- 集群状态信息 --- cluster_state:fail cluster_slots_assigned:16384 cluster_slots_ok:0 cluster_slots_pfail:16384 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:4 cluster_stats_messages_ping_sent:5 cluster_stats_messages_sent:5 cluster_stats_messages_received:0 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave,fail? f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 1756699149870 1756699147353 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave,fail? d4512197faed320ad44935144bd1e67f8e4e1925 1756699149870 1756699147353 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master,fail? - 1756699148363 1756699147353 4 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave,fail? f49b9ec6a80833cb30167969e9f74558c9513f47 1756699149367 1756699147353 7 disconnected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master,fail? - 1756699149870 1756699147353 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:fail cluster_slots_assigned:16384 cluster_slots_ok:5461 cluster_slots_pfail:10923 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:1 cluster_stats_messages_ping_sent:5 cluster_stats_messages_sent:5 cluster_stats_messages_received:0 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master,fail? - 1756699149780 1756699147258 4 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master,fail? - 1756699149780 1756699147258 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave,fail? d4512197faed320ad44935144bd1e67f8e4e1925 1756699149278 1756699147258 4 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave,fail? f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 1756699149780 1756699147258 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master,fail? - 1756699148274 1756699147258 1 connected 0-5460 --- 集群状态信息 --- cluster_state:fail cluster_slots_assigned:16384 cluster_slots_ok:0 cluster_slots_pfail:16384 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:7 cluster_stats_messages_ping_sent:5 cluster_stats_messages_sent:5 cluster_stats_messages_received:0 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 警告: 节点 rediscl-vmwsrc-shard-4wn-0 报告集群状态为 'fail'，期望值为 'ok' 警告: 节点 rediscl-vmwsrc-shard-4wn-0 报告正常槽位数为 '0'，期望与分配的槽位数相同 警告: 节点 rediscl-vmwsrc-shard-4wn-0 报告 16384 个疑似故障槽位和 0 个故障槽位 警告: 节点 rediscl-vmwsrc-shard-4wn-0 报告集群中存在故障或疑似故障节点 警告: 节点 rediscl-vmwsrc-shard-7wk-0 报告集群状态为 'fail'，期望值为 'ok' 警告: 节点 rediscl-vmwsrc-shard-7wk-0 报告正常槽位数为 '5461'，期望与分配的槽位数相同 警告: 节点 rediscl-vmwsrc-shard-7wk-0 报告 10923 个疑似故障槽位和 0 个故障槽位 警告: 节点 rediscl-vmwsrc-shard-7wk-0 报告集群中存在故障或疑似故障节点 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告集群状态为 'fail'，期望值为 'ok' 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告正常槽位数为 '0'，期望与分配的槽位数相同 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告 16384 个疑似故障槽位和 0 个故障槽位 警告: 节点 rediscl-vmwsrc-shard-d9g-0 报告集群中存在故障或疑似故障节点 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 主节点(故障) - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点(故障) - 0-5460 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 从节点(故障) f49b9ec6a80833cb30167969e9f74558c9513f47 b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579 从节点(故障) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点(故障) - 5461-10922 (共 0 个槽位) 检测到 5 个故障或疑似故障节点: ID IP地址 状态 所属分片 报告时间 ---------------------------------------- ------------- ---------------- ----------- ------------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 主节点(故障) 未知 16383 (epoch) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点(故障) 7wk 5460 (epoch) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 从节点(故障) d9g 未知 b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579 从节点(故障) 未知 未知 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点(故障) 未知 10922 (epoch) 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 警告: 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 的故障状态在不同视图中不一致 参考节点视图: fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected (正常) 当前节点视图: fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave,fail? d4512197faed320ad44935144bd1e67f8e4e1925 1756699149870 1756699147353 4 connected (fail) 警告: 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 的故障状态在不同视图中不一致 参考节点视图: f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master,fail? - 1756699149033 1756699146499 1 connected 0-5460 (fail) 当前节点视图: f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 (正常) 警告: 发现 2 个节点的角色、主从关系或故障状态不一致 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 警告: 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 的故障状态在不同视图中不一致 参考节点视图: fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected (正常) 当前节点视图: fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave,fail? d4512197faed320ad44935144bd1e67f8e4e1925 1756699149278 1756699147258 4 connected (fail) 警告: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b 的故障状态在不同视图中不一致 参考节点视图: 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave,fail? f49b9ec6a80833cb30167969e9f74558c9513f47 1756699149033 1756699146499 7 connected (fail) 当前节点视图: 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected (正常) 警告: 发现 2 个节点的角色、主从关系或故障状态不一致 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 f49b9ec6a80833cb30167969e9f74558c9513f47 (10.224.0.9:32177) 负责 0 个槽位: 10923-16383 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 5461-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 0 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 0 个从节点, 1 个故障节点 ❌ 分片 d9g: 0 个主节点, 1 个从节点, 1 个故障节点 ❌ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 5 个故障或疑似故障节点 - 集群状态统一显示为OK: 否 - 所有槽位正确分配: 否 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 否 集群健康状态: 不健康 ❌ 检测到以下问题: - 集群状态异常: 一个或多个节点报告集群状态不为OK - 槽位分配问题: 槽位数量不正确或存在故障槽位 - 存在故障节点: 检测到一个或多个节点被标记为故障或疑似故障 建议措施: 1. 检查故障节点的网络连接和运行状态 2. 对于故障从节点，考虑重启或替换它们 3. 对于故障主节点，确认是否已自动进行故障转移 health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699376375 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699377583 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699378388 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699377000 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699377382 4 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:4 cluster_stats_messages_ping_sent:357 cluster_stats_messages_pong_sent:373 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:741 cluster_stats_messages_ping_received:373 cluster_stats_messages_pong_received:353 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:734 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699379684 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699378576 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699378000 4 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699379180 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699379079 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:4 cluster_stats_messages_ping_sent:350 cluster_stats_messages_pong_sent:327 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:687 cluster_stats_messages_ping_received:325 cluster_stats_messages_pong_received:348 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:2 cluster_stats_messages_received:677 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699379215 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699380728 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699379718 4 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699380223 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699380000 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:1 cluster_stats_messages_ping_sent:366 cluster_stats_messages_pong_sent:373 cluster_stats_messages_meet_sent:2 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:743 cluster_stats_messages_ping_received:372 cluster_stats_messages_pong_received:363 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:745 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699381584 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699380000 4 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699381000 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699380000 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699381988 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:1 cluster_stats_messages_ping_sent:346 cluster_stats_messages_pong_sent:330 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:687 cluster_stats_messages_ping_received:330 cluster_stats_messages_pong_received:345 cluster_stats_messages_fail_received:1 cluster_stats_messages_received:676 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699382000 4 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699381000 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699382000 4 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699383303 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699382297 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:7 cluster_stats_messages_ping_sent:374 cluster_stats_messages_pong_sent:388 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:768 cluster_stats_messages_ping_received:388 cluster_stats_messages_pong_received:370 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:767 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699382593 7 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699384405 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699384506 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699384000 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699382391 4 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:7 cluster_stats_messages_ping_sent:332 cluster_stats_messages_pong_sent:317 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:659 cluster_stats_messages_ping_received:315 cluster_stats_messages_pong_received:330 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:649 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 主节点 - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 从节点 f49b9ec6a80833cb30167969e9f74558c9513f47 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点 - 5461-10922 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 f49b9ec6a80833cb30167969e9f74558c9513f47 (10.224.0.9:32177) 负责 0 个槽位: 10923-16383 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 5461-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl get pvc -l app.kubernetes.io/instance=rediscl-vmwsrc,apps.kubeblocks.io/component-name=shard,apps.kubeblocks.io/vct-name=data --namespace ns-pwglh ` No resources found in ns-pwglh namespace. rediscl-vmwsrc shard data pvc is empty cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in rediscl-vmwsrc namespace. `kbcli cluster volume-expand rediscl-vmwsrc --auto-approve --force=true --components shard --volume-claim-templates data --storage 6Gi --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-volumeexpansion-6flxj created successfully, you can view the progress: kbcli cluster describe-ops rediscl-vmwsrc-volumeexpansion-6flxj -n ns-pwglh check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-volumeexpansion-6flxj ns-pwglh VolumeExpansion rediscl-vmwsrc shard Creating -/- Sep 01,2025 12:04 UTC+0800 check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Updating Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-1;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699604523 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 4 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699605228 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699604221 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699604523 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699604725 4 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:4 cluster_stats_messages_ping_sent:771 cluster_stats_messages_pong_sent:785 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1567 cluster_stats_messages_ping_received:785 cluster_stats_messages_pong_received:767 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:1560 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699606000 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699605479 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699606153 4 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699606487 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699606588 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:4 cluster_stats_messages_ping_sent:764 cluster_stats_messages_pong_sent:750 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:1524 cluster_stats_messages_ping_received:748 cluster_stats_messages_pong_received:762 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:2 cluster_stats_messages_received:1514 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699607000 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699608002 4 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699606000 4 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699606000 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699607000 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:1 cluster_stats_messages_ping_sent:767 cluster_stats_messages_pong_sent:785 cluster_stats_messages_meet_sent:2 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:1556 cluster_stats_messages_ping_received:784 cluster_stats_messages_pong_received:764 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:1558 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699608588 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699608588 4 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699607000 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699608990 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699608000 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:1 cluster_stats_messages_ping_sent:759 cluster_stats_messages_pong_sent:728 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:1498 cluster_stats_messages_ping_received:728 cluster_stats_messages_pong_received:758 cluster_stats_messages_fail_received:1 cluster_stats_messages_received:1487 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699610365 4 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699609000 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699608351 4 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699609358 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699610000 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:7 cluster_stats_messages_ping_sent:779 cluster_stats_messages_pong_sent:792 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1577 cluster_stats_messages_ping_received:792 cluster_stats_messages_pong_received:775 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:1576 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699611603 7 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699611100 4 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699609588 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699610000 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756699611000 4 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:7 cluster_stats_messages_ping_sent:739 cluster_stats_messages_pong_sent:721 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:1470 cluster_stats_messages_ping_received:719 cluster_stats_messages_pong_received:737 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:1460 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 主节点 - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 从节点 f49b9ec6a80833cb30167969e9f74558c9513f47 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点 - 5461-10922 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 f49b9ec6a80833cb30167969e9f74558c9513f47 (10.224.0.9:32177) 负责 0 个槽位: 10923-16383 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 5461-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success No resources found in rediscl-vmwsrc namespace. check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-volumeexpansion-6flxj ns-pwglh VolumeExpansion rediscl-vmwsrc shard Succeed 6/6 Sep 01,2025 12:04 UTC+0800 check ops status done ops_status:rediscl-vmwsrc-volumeexpansion-6flxj ns-pwglh VolumeExpansion rediscl-vmwsrc shard Succeed 6/6 Sep 01,2025 12:04 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests rediscl-vmwsrc-volumeexpansion-6flxj --namespace ns-pwglh ` opsrequest.apps.kubeblocks.io/rediscl-vmwsrc-volumeexpansion-6flxj patched `kbcli cluster delete-ops --name rediscl-vmwsrc-volumeexpansion-6flxj --force --auto-approve --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-volumeexpansion-6flxj deleted test failover podfailure check cluster status before cluster-failover-podfailure check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-rediscl-vmwsrc" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-rediscl-vmwsrc" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podfailure-rediscl-vmwsrc namespace: ns-pwglh spec: selector: namespaces: - ns-pwglh labelSelectors: apps.kubeblocks.io/pod-name: rediscl-vmwsrc-shard-4wn-1 mode: all action: pod-failure duration: 2m `kubectl apply -f test-chaos-mesh-podfailure-rediscl-vmwsrc.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podfailure-rediscl-vmwsrc created apply test-chaos-mesh-podfailure-rediscl-vmwsrc.yaml Success `rm -rf test-chaos-mesh-podfailure-rediscl-vmwsrc.yaml` podfailure chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Abnormal Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Abnormal cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:59 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-1;secondary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699820000 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 14 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699819088 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699819000 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699820094 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756699820599 14 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:14 cluster_my_epoch:14 cluster_stats_messages_ping_sent:1132 cluster_stats_messages_pong_sent:1165 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-req_sent:10 cluster_stats_messages_sent:2313 cluster_stats_messages_ping_received:1159 cluster_stats_messages_pong_received:1127 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:2298 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699821580 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699821379 14 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699821000 7 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 14 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699820000 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699821000 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:14 cluster_my_epoch:14 cluster_stats_messages_ping_sent:49 cluster_stats_messages_pong_sent:28 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:78 cluster_stats_messages_ping_received:28 cluster_stats_messages_pong_received:50 cluster_stats_messages_received:78 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699822000 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699822686 14 connected 5461-10922 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756699820573 14 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699821579 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699821000 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:14 cluster_my_epoch:1 cluster_stats_messages_ping_sent:1123 cluster_stats_messages_pong_sent:1132 cluster_stats_messages_meet_sent:2 cluster_stats_messages_auth-ack_sent:3 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:2261 cluster_stats_messages_ping_received:1131 cluster_stats_messages_pong_received:1120 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:8 cluster_stats_messages_auth-req_received:3 cluster_stats_messages_received:2263 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756699823000 14 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699822000 14 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699824153 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699823570 7 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699823671 7 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:14 cluster_my_epoch:1 cluster_stats_messages_ping_sent:1120 cluster_stats_messages_pong_sent:1094 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:2231 cluster_stats_messages_ping_received:1094 cluster_stats_messages_pong_received:1119 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:2215 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756699824175 14 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756699823168 7 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699824677 14 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 0 7 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699825181 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699824074 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:14 cluster_my_epoch:7 cluster_stats_messages_ping_sent:1139 cluster_stats_messages_pong_sent:1156 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:2302 cluster_stats_messages_ping_received:1156 cluster_stats_messages_pong_received:1135 cluster_stats_messages_fail_received:8 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:2302 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 7 connected 10923-16383 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave f49b9ec6a80833cb30167969e9f74558c9513f47 0 1756699826000 7 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756699826238 14 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756699825000 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756699825231 1 connected 0-5460 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756699825231 14 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:14 cluster_my_epoch:7 cluster_stats_messages_ping_sent:1094 cluster_stats_messages_pong_sent:1081 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:2187 cluster_stats_messages_ping_received:1079 cluster_stats_messages_pong_received:1092 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:2177 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 主节点 - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 主节点 - 5461-10922 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 从节点 f49b9ec6a80833cb30167969e9f74558c9513f47 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 f49b9ec6a80833cb30167969e9f74558c9513f47 (10.224.0.9:32177) 负责 0 个槽位: 10923-16383 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.9:30435) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-rediscl-vmwsrc" force deleted podchaos.chaos-mesh.org/test-chaos-mesh-podfailure-rediscl-vmwsrc patched check failover pod name failover pod name:rediscl-vmwsrc-shard-4wn-0 failover podfailure Success cluster stop check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster stop rediscl-vmwsrc --auto-approve --force=true --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-stop-8t7pv created successfully, you can view the progress: kbcli cluster describe-ops rediscl-vmwsrc-stop-8t7pv -n ns-pwglh check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-stop-8t7pv ns-pwglh Stop rediscl-vmwsrc Creating -/- Sep 01,2025 12:11 UTC+0800 check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Stopped Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc check cluster status done cluster_status:Stopped check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME check pod status done check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-stop-8t7pv ns-pwglh Stop rediscl-vmwsrc shard Succeed 6/6 Sep 01,2025 12:11 UTC+0800 check ops status done ops_status:rediscl-vmwsrc-stop-8t7pv ns-pwglh Stop rediscl-vmwsrc shard Succeed 6/6 Sep 01,2025 12:11 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests rediscl-vmwsrc-stop-8t7pv --namespace ns-pwglh ` opsrequest.apps.kubeblocks.io/rediscl-vmwsrc-stop-8t7pv patched `kbcli cluster delete-ops --name rediscl-vmwsrc-stop-8t7pv --force --auto-approve --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-stop-8t7pv deleted cluster start check cluster status before ops check cluster status done cluster_status:Stopped `kbcli cluster start rediscl-vmwsrc --force=true --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-start-vb967 created successfully, you can view the progress: kbcli cluster describe-ops rediscl-vmwsrc-start-vb967 -n ns-pwglh check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-start-vb967 ns-pwglh Start rediscl-vmwsrc Creating -/- Sep 01,2025 12:11 UTC+0800 check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Updating Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 12:11 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:12 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 12:11 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 12:12 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:11 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:12 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700017565 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700016860 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700017565 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700016558 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 14 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700015853 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:16 cluster_my_epoch:14 cluster_stats_messages_ping_sent:112 cluster_stats_messages_pong_sent:130 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:16 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:262 cluster_stats_messages_ping_received:128 cluster_stats_messages_pong_received:110 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:10 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:252 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700018852 16 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 14 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700018000 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700018550 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700018000 14 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700017540 16 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:16 cluster_my_epoch:14 cluster_stats_messages_ping_sent:91 cluster_stats_messages_pong_sent:84 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:187 cluster_stats_messages_ping_received:84 cluster_stats_messages_pong_received:90 cluster_stats_messages_fail_received:1 cluster_stats_messages_received:175 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700019000 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700019000 14 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700019521 16 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700019521 16 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700019822 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:16 cluster_my_epoch:1 cluster_stats_messages_ping_sent:115 cluster_stats_messages_pong_sent:126 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:16 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:261 cluster_stats_messages_ping_received:124 cluster_stats_messages_pong_received:113 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:246 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700019615 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700019000 16 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700019513 14 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700020622 16 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700019000 14 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:16 cluster_my_epoch:1 cluster_stats_messages_ping_sent:91 cluster_stats_messages_pong_sent:74 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:171 cluster_stats_messages_ping_received:74 cluster_stats_messages_pong_received:91 cluster_stats_messages_received:165 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700021860 16 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700020537 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700020537 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700021558 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700020854 14 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:16 cluster_my_epoch:16 cluster_stats_messages_ping_sent:119 cluster_stats_messages_pong_sent:124 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:14 cluster_stats_messages_auth-req_sent:10 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:269 cluster_stats_messages_ping_received:118 cluster_stats_messages_pong_received:116 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:245 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700021590 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700021000 14 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700021590 16 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700022596 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700022000 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:16 cluster_my_epoch:16 cluster_stats_messages_ping_sent:85 cluster_stats_messages_pong_sent:68 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:154 cluster_stats_messages_ping_received:65 cluster_stats_messages_pong_received:85 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:153 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 10923-16383 (共 0 个槽位) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.10:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.10:30435 主节点 - 5461-10922 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.10:30971 主节点 - 0-5460 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 10923-16383 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.10:30435) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.10:30971) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-start-vb967 ns-pwglh Start rediscl-vmwsrc shard Succeed 6/6 Sep 01,2025 12:11 UTC+0800 check ops status done ops_status:rediscl-vmwsrc-start-vb967 ns-pwglh Start rediscl-vmwsrc shard Succeed 6/6 Sep 01,2025 12:11 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests rediscl-vmwsrc-start-vb967 --namespace ns-pwglh ` opsrequest.apps.kubeblocks.io/rediscl-vmwsrc-start-vb967 patched `kbcli cluster delete-ops --name rediscl-vmwsrc-start-vb967 --force --auto-approve --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-start-vb967 deleted test failover drainnode check cluster status before cluster-failover-drainnode check cluster status done cluster_status:Running check node drain check node drain success kubectl get pod rediscl-vmwsrc-shard-4wn-0 --namespace ns-pwglh -o jsonpath='***.spec.nodeName***' get node name:aks-cicdamdpool-68727890-vmss000006 success check if multiple pods are on the same node kubectl get pod rediscl-vmwsrc-shard-4wn-1 --namespace ns-pwglh -o jsonpath='***.spec.nodeName***' get node name:aks-cicdamdpool-68727890-vmss000008 success kubectl drain aks-cicdamdpool-68727890-vmss000006 --delete-emptydir-data --ignore-daemonsets --force --grace-period 0 --timeout 60s node/aks-cicdamdpool-68727890-vmss000006 cordoned Warning: ignoring DaemonSet-managed Pods: chaos-mesh/chaos-daemon-ppjfv, kb-luwux/kb-addon-apecloud-otel-collector-jglbc, kube-system/azure-cns-9wczd, kube-system/azure-ip-masq-agent-pwvhd, kube-system/cloud-node-manager-69j8w, kube-system/csi-azuredisk-node-xslv5, kube-system/csi-azurefile-node-rl5r7, kube-system/kube-proxy-lczjv evicting pod ns-pwglh/rediscl-vmwsrc-shard-7wk-1 evicting pod ns-bwwyh/kb-post-provision-job-goldendb-fijseo-goldendb-dn-00-z9mp8 evicting pod ns-pwglh/rediscl-vmwsrc-shard-4wn-0 evicting pod ns-drhff/vanilla-ewmtgg-postgresql-1 evicting pod ns-pwglh/rediscl-vmwsrc-shard-7wk-0 evicting pod ns-bwwyh/kb-post-provision-job-goldendb-fijseo-goldendb-dn-00-t6c2b evicting pod ns-bwwyh/kb-post-provision-job-goldendb-fijseo-goldendb-dn-00-qctzp evicting pod ns-bwwyh/kb-post-provision-job-goldendb-fijseo-goldendb-dn-00-ldjwx pod/kb-post-provision-job-goldendb-fijseo-goldendb-dn-00-ldjwx evicted pod/kb-post-provision-job-goldendb-fijseo-goldendb-dn-00-t6c2b evicted pod/rediscl-vmwsrc-shard-4wn-0 evicted pod/kb-post-provision-job-goldendb-fijseo-goldendb-dn-00-z9mp8 evicted pod/vanilla-ewmtgg-postgresql-1 evicted pod/kb-post-provision-job-goldendb-fijseo-goldendb-dn-00-qctzp evicted pod/rediscl-vmwsrc-shard-7wk-0 evicted pod/rediscl-vmwsrc-shard-7wk-1 evicted node/aks-cicdamdpool-68727890-vmss000006 drained kubectl uncordon aks-cicdamdpool-68727890-vmss000006 node/aks-cicdamdpool-68727890-vmss000006 uncordoned check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Updating Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:14 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:12 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:14 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:15 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:11 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:12 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 14 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700182000 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700182770 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700182000 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700182000 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700182267 14 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:14 cluster_stats_messages_ping_sent:101 cluster_stats_messages_pong_sent:84 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:197 cluster_stats_messages_ping_received:83 cluster_stats_messages_pong_received:101 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:188 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700182263 16 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 14 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700182000 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700182000 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700183270 14 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700182566 16 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:14 cluster_stats_messages_ping_sent:340 cluster_stats_messages_pong_sent:345 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:14 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:706 cluster_stats_messages_ping_received:345 cluster_stats_messages_pong_received:336 cluster_stats_messages_fail_received:8 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:690 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700185280 14 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700184000 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700184273 16 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700184575 14 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700184000 16 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:1 cluster_stats_messages_ping_sent:101 cluster_stats_messages_pong_sent:98 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:211 cluster_stats_messages_ping_received:96 cluster_stats_messages_pong_received:100 cluster_stats_messages_meet_received:2 cluster_stats_messages_received:198 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700184575 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700185000 14 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700184000 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700184575 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700185885 16 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:1 cluster_stats_messages_ping_sent:60 cluster_stats_messages_pong_sent:34 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:95 cluster_stats_messages_ping_received:34 cluster_stats_messages_pong_received:61 cluster_stats_messages_received:95 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700187000 16 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700186593 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700186000 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700187097 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700187601 14 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:16 cluster_stats_messages_ping_sent:384 cluster_stats_messages_pong_sent:401 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:22 cluster_stats_messages_auth-req_sent:10 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:820 cluster_stats_messages_ping_received:393 cluster_stats_messages_pong_received:378 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:13 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:791 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700187000 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700187505 14 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700187908 16 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700186901 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700186599 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:16 cluster_stats_messages_ping_sent:344 cluster_stats_messages_pong_sent:330 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:680 cluster_stats_messages_ping_received:327 cluster_stats_messages_pong_received:341 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:679 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 主节点 - 5461-10922 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 10923-16383 (共 0 个槽位) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.9:30435) 负责 0 个槽位: 5461-10922 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 10923-16383 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success check failover pod name failover pod name:rediscl-vmwsrc-shard-4wn-0 failover drainnode Success test failover kill1 check cluster status before cluster-failover-kill1 check cluster status done cluster_status:Running check node drain check node drain success `kill 1` Defaulted container "redis-cluster" out of: redis-cluster, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file exec return message: check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Updating Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:14 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:12 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:14 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:15 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:11 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:12 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 14 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700285512 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700286518 14 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700284505 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700286518 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700285512 16 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:14 cluster_stats_messages_ping_sent:50 cluster_stats_messages_pong_sent:26 cluster_stats_messages_meet_sent:2 cluster_stats_messages_sent:78 cluster_stats_messages_ping_received:26 cluster_stats_messages_pong_received:52 cluster_stats_messages_received:78 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700287198 16 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 14 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700287601 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700286594 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700286695 14 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700287701 16 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:14 cluster_stats_messages_ping_sent:526 cluster_stats_messages_pong_sent:528 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:14 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:1075 cluster_stats_messages_ping_received:528 cluster_stats_messages_pong_received:521 cluster_stats_messages_fail_received:8 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:1058 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700287000 14 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700288003 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700287000 16 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700286594 14 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700287000 16 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:1 cluster_stats_messages_ping_sent:282 cluster_stats_messages_pong_sent:280 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:574 cluster_stats_messages_ping_received:277 cluster_stats_messages_pong_received:280 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:560 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700288367 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700289000 14 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700289374 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700288000 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700289575 16 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:1 cluster_stats_messages_ping_sent:242 cluster_stats_messages_pong_sent:215 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:458 cluster_stats_messages_ping_received:215 cluster_stats_messages_pong_received:242 cluster_stats_messages_received:457 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700290828 16 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700290626 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700289000 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700290000 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700289000 14 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:16 cluster_stats_messages_ping_sent:558 cluster_stats_messages_pong_sent:587 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:22 cluster_stats_messages_auth-req_sent:10 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:1180 cluster_stats_messages_ping_received:578 cluster_stats_messages_pong_received:551 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:13 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:1150 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700290582 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700291589 14 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700291000 16 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700291589 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700291690 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:16 cluster_stats_messages_ping_sent:522 cluster_stats_messages_pong_sent:520 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:1048 cluster_stats_messages_ping_received:517 cluster_stats_messages_pong_received:518 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:1046 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 主节点 - 5461-10922 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 10923-16383 (共 0 个槽位) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.9:30435) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 10923-16383 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success check failover pod name failover pod name:rediscl-vmwsrc-shard-4wn-0 failover kill1 Success cluster update enable-all-logs `kbcli cluster update rediscl-vmwsrc --enable-all-logs=true --namespace ns-pwglh ` error: name is required check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Running Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:14 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:12 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:14 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:15 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:11 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:12 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 14 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700369159 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700368656 14 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700368555 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700368000 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700368153 16 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:14 cluster_stats_messages_ping_sent:195 cluster_stats_messages_pong_sent:174 cluster_stats_messages_meet_sent:2 cluster_stats_messages_sent:371 cluster_stats_messages_ping_received:174 cluster_stats_messages_pong_received:197 cluster_stats_messages_received:371 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700369000 16 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 14 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700369550 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700368541 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700369047 14 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700370054 16 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:14 cluster_stats_messages_ping_sent:671 cluster_stats_messages_pong_sent:679 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:14 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:1371 cluster_stats_messages_ping_received:679 cluster_stats_messages_pong_received:666 cluster_stats_messages_fail_received:8 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:1354 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700371594 14 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700371092 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700371000 16 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700371000 14 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700371092 16 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:1 cluster_stats_messages_ping_sent:429 cluster_stats_messages_pong_sent:424 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:865 cluster_stats_messages_ping_received:421 cluster_stats_messages_pong_received:427 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:851 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700372000 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700373567 14 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700373000 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700372000 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700373163 16 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:1 cluster_stats_messages_ping_sent:391 cluster_stats_messages_pong_sent:359 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:751 cluster_stats_messages_ping_received:359 cluster_stats_messages_pong_received:391 cluster_stats_messages_received:750 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700373000 16 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700374561 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700373556 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700373555 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700374058 14 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:16 cluster_stats_messages_ping_sent:703 cluster_stats_messages_pong_sent:739 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:22 cluster_stats_messages_auth-req_sent:10 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:1477 cluster_stats_messages_ping_received:730 cluster_stats_messages_pong_received:696 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:13 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:1447 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700375000 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700374000 14 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700375000 16 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700375303 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700374801 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:16 cluster_stats_messages_ping_sent:677 cluster_stats_messages_pong_sent:672 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:1355 cluster_stats_messages_ping_received:669 cluster_stats_messages_pong_received:673 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:1353 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 主节点 - 5461-10922 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 10923-16383 (共 0 个槽位) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.9:30435) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 10923-16383 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success test failover networkduplicate check cluster status before cluster-failover-networkduplicate check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-rediscl-vmwsrc" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-rediscl-vmwsrc" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkduplicate-rediscl-vmwsrc namespace: ns-pwglh spec: selector: namespaces: - ns-pwglh labelSelectors: apps.kubeblocks.io/pod-name: rediscl-vmwsrc-shard-4wn-0 mode: all action: duplicate duplicate: duplicate: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkduplicate-rediscl-vmwsrc.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkduplicate-rediscl-vmwsrc created apply test-chaos-mesh-networkduplicate-rediscl-vmwsrc.yaml Success `rm -rf test-chaos-mesh-networkduplicate-rediscl-vmwsrc.yaml` networkduplicate chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Running Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:14 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:12 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:14 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:15 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:11 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:12 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 14 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700559509 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700559510 14 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700558000 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700559610 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700558603 16 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:14 cluster_stats_messages_ping_sent:536 cluster_stats_messages_pong_sent:521 cluster_stats_messages_meet_sent:2 cluster_stats_messages_sent:1059 cluster_stats_messages_ping_received:521 cluster_stats_messages_pong_received:538 cluster_stats_messages_received:1059 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700560520 16 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 14 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700559007 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700561025 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700561000 14 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700560520 16 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:14 cluster_stats_messages_ping_sent:1009 cluster_stats_messages_pong_sent:1016 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:14 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:2046 cluster_stats_messages_ping_received:1016 cluster_stats_messages_pong_received:1004 cluster_stats_messages_fail_received:8 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:2029 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700561000 14 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700561630 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700560623 16 connected 10923-16383 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700561528 14 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700560000 16 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:1 cluster_stats_messages_ping_sent:767 cluster_stats_messages_pong_sent:760 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:1539 cluster_stats_messages_ping_received:757 cluster_stats_messages_pong_received:765 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:1525 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700561000 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700562936 14 connected 5461-10922 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700561929 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700562000 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700562000 16 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:1 cluster_stats_messages_ping_sent:729 cluster_stats_messages_pong_sent:686 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1416 cluster_stats_messages_ping_received:686 cluster_stats_messages_pong_received:729 cluster_stats_messages_received:1415 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700564000 16 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700562000 1 connected 0-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700563586 1 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700563989 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700563586 14 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:16 cluster_stats_messages_ping_sent:1041 cluster_stats_messages_pong_sent:1074 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:22 cluster_stats_messages_auth-req_sent:10 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:2150 cluster_stats_messages_ping_received:1065 cluster_stats_messages_pong_received:1034 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:13 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:2120 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700564044 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700563843 14 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700564547 16 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700564849 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700564548 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:17 cluster_my_epoch:16 cluster_stats_messages_ping_sent:1022 cluster_stats_messages_pong_sent:1023 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:2051 cluster_stats_messages_ping_received:1020 cluster_stats_messages_pong_received:1018 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:2049 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 主节点 - 5461-10922 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 10923-16383 (共 0 个槽位) f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.9:30435) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 10923-16383 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-rediscl-vmwsrc" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-rediscl-vmwsrc" not found check failover pod name failover pod name:rediscl-vmwsrc-shard-4wn-0 failover networkduplicate Success cluster vscale check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale rediscl-vmwsrc --auto-approve --force=true --components shard --cpu 200m --memory 0.6Gi --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-verticalscaling-bb8ld created successfully, you can view the progress: kbcli cluster describe-ops rediscl-vmwsrc-verticalscaling-bb8ld -n ns-pwglh check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-verticalscaling-bb8ld ns-pwglh VerticalScaling rediscl-vmwsrc shard Creating -/- Sep 01,2025 12:23 UTC+0800 check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Updating Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700704513 16 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 14 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700704513 16 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700703809 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700703103 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700704814 14 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:14 cluster_stats_messages_ping_sent:53 cluster_stats_messages_pong_sent:38 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:99 cluster_stats_messages_ping_received:36 cluster_stats_messages_pong_received:53 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:94 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700704794 16 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700705601 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700704000 16 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700705803 14 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700704592 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:14 cluster_stats_messages_ping_sent:100 cluster_stats_messages_pong_sent:88 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_sent:204 cluster_stats_messages_ping_received:88 cluster_stats_messages_pong_received:96 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:188 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700706000 14 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700706528 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700706830 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700706000 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700705824 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:1 cluster_stats_messages_ping_sent:55 cluster_stats_messages_pong_sent:44 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:106 cluster_stats_messages_ping_received:42 cluster_stats_messages_pong_received:55 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:102 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700708529 14 connected 5461-10922 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700707520 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700707000 16 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700707000 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700707923 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:1 cluster_stats_messages_ping_sent:103 cluster_stats_messages_pong_sent:104 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:223 cluster_stats_messages_ping_received:104 cluster_stats_messages_pong_received:99 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:208 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700708000 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700709535 14 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700708528 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 16 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700709741 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700708730 14 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:16 cluster_stats_messages_ping_sent:64 cluster_stats_messages_pong_sent:42 cluster_stats_messages_meet_sent:2 cluster_stats_messages_sent:108 cluster_stats_messages_ping_received:40 cluster_stats_messages_pong_received:63 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:1 cluster_stats_messages_received:106 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700710900 14 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700710000 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700709000 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700709590 16 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700709892 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:16 cluster_stats_messages_ping_sent:106 cluster_stats_messages_pong_sent:96 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:218 cluster_stats_messages_ping_received:96 cluster_stats_messages_pong_received:102 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:203 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 主节点 - 5461-10922 (共 0 个槽位) 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 10923-16383 (共 0 个槽位) b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.9:30435) 负责 0 个槽位: 5461-10922 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 10923-16383 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success check ops status `kbcli cluster list-ops rediscl-vmwsrc --status all --namespace ns-pwglh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-vmwsrc-verticalscaling-bb8ld ns-pwglh VerticalScaling rediscl-vmwsrc shard Succeed 6/6 Sep 01,2025 12:23 UTC+0800 check ops status done ops_status:rediscl-vmwsrc-verticalscaling-bb8ld ns-pwglh VerticalScaling rediscl-vmwsrc shard Succeed 6/6 Sep 01,2025 12:23 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests rediscl-vmwsrc-verticalscaling-bb8ld --namespace ns-pwglh ` opsrequest.apps.kubeblocks.io/rediscl-vmwsrc-verticalscaling-bb8ld patched `kbcli cluster delete-ops --name rediscl-vmwsrc-verticalscaling-bb8ld --force --auto-approve --namespace ns-pwglh ` OpsRequest rediscl-vmwsrc-verticalscaling-bb8ld deleted test failover oom check cluster status before cluster-failover-oom check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-rediscl-vmwsrc" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-rediscl-vmwsrc" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-rediscl-vmwsrc namespace: ns-pwglh spec: selector: namespaces: - ns-pwglh labelSelectors: apps.kubeblocks.io/pod-name: rediscl-vmwsrc-shard-4wn-0 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-rediscl-vmwsrc.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-rediscl-vmwsrc created apply test-chaos-mesh-oom-rediscl-vmwsrc.yaml Success `rm -rf test-chaos-mesh-oom-rediscl-vmwsrc.yaml` check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Updating Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700796580 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700797587 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700797000 16 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 14 connected 5461-10922 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700797000 14 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700797989 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:14 cluster_stats_messages_ping_sent:48 cluster_stats_messages_pong_sent:25 cluster_stats_messages_meet_sent:2 cluster_stats_messages_sent:75 cluster_stats_messages_ping_received:25 cluster_stats_messages_pong_received:50 cluster_stats_messages_received:75 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700799000 16 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700798486 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700799089 16 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700799491 14 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700799000 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:14 cluster_stats_messages_ping_sent:268 cluster_stats_messages_pong_sent:256 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_sent:540 cluster_stats_messages_ping_received:256 cluster_stats_messages_pong_received:263 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:523 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700799599 14 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700799599 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700799498 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700799000 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700800506 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:1 cluster_stats_messages_ping_sent:217 cluster_stats_messages_pong_sent:212 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:436 cluster_stats_messages_ping_received:209 cluster_stats_messages_pong_received:216 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:431 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700800000 14 connected 5461-10922 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700801589 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700801590 16 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700800582 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700801589 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:1 cluster_stats_messages_ping_sent:258 cluster_stats_messages_pong_sent:279 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:553 cluster_stats_messages_ping_received:279 cluster_stats_messages_pong_received:253 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:537 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700803604 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700803000 14 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700803000 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 16 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700803403 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700802898 14 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:16 cluster_stats_messages_ping_sent:239 cluster_stats_messages_pong_sent:201 cluster_stats_messages_meet_sent:2 cluster_stats_messages_sent:442 cluster_stats_messages_ping_received:198 cluster_stats_messages_pong_received:237 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:1 cluster_stats_messages_received:439 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700804608 14 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700803601 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700803601 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700804000 16 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700803096 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:16 cluster_stats_messages_ping_sent:263 cluster_stats_messages_pong_sent:262 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:541 cluster_stats_messages_ping_received:262 cluster_stats_messages_pong_received:258 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:525 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 主节点 - 5461-10922 (共 0 个槽位) d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 10923-16383 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.9:30435) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-rediscl-vmwsrc" force deleted stresschaos.chaos-mesh.org/test-chaos-mesh-oom-rediscl-vmwsrc patched check failover pod name failover pod name:rediscl-vmwsrc-shard-4wn-0 failover oom Success patch rediscl-vmwsrc shards 4 `kubectl patch cluster rediscl-vmwsrc --namespace ns-pwglh --type json -p '[***"op": "replace", "path": "/spec/shardingSpecs/0/shards", "value": '4'***]'` cluster.apps.kubeblocks.io/rediscl-vmwsrc patched check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Running Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700877551 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700877551 1 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700876543 16 connected 10923-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 14 connected 5461-10922 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700876000 14 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700876543 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:14 cluster_stats_messages_ping_sent:183 cluster_stats_messages_pong_sent:167 cluster_stats_messages_meet_sent:2 cluster_stats_messages_sent:352 cluster_stats_messages_ping_received:167 cluster_stats_messages_pong_received:185 cluster_stats_messages_received:352 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700878000 16 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700878054 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700877000 16 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700877551 14 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700878000 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:14 cluster_stats_messages_ping_sent:412 cluster_stats_messages_pong_sent:390 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_sent:818 cluster_stats_messages_ping_received:390 cluster_stats_messages_pong_received:407 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:801 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700879564 14 connected 5461-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700878557 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700879060 16 connected 10923-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700879000 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700880067 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:1 cluster_stats_messages_ping_sent:361 cluster_stats_messages_pong_sent:358 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:726 cluster_stats_messages_ping_received:355 cluster_stats_messages_pong_received:360 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:721 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700879161 14 connected 5461-10922 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700881175 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700881000 16 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700880168 1 connected 0-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700880000 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:1 cluster_stats_messages_ping_sent:402 cluster_stats_messages_pong_sent:430 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:848 cluster_stats_messages_ping_received:430 cluster_stats_messages_pong_received:397 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:832 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756700880973 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700881074 14 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700881980 1 connected 0-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 16 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700880571 1 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700880000 14 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:16 cluster_stats_messages_ping_sent:382 cluster_stats_messages_pong_sent:346 cluster_stats_messages_meet_sent:2 cluster_stats_messages_sent:730 cluster_stats_messages_ping_received:343 cluster_stats_messages_pong_received:380 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:1 cluster_stats_messages_received:727 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756700882181 14 connected 5461-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700882685 1 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700881074 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756700883000 16 connected 10923-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700883188 1 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:16 cluster_stats_messages_ping_sent:409 cluster_stats_messages_pong_sent:401 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:826 cluster_stats_messages_ping_received:401 cluster_stats_messages_pong_received:404 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:810 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-w8k-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-w8k, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-w8k-account-default 密码获取成功，前缀: 55... 无法从环境变量中获取Redis端口，使用默认端口6379 --- 集群节点信息 --- 执行命令失败 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 10923-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 主节点 - 5461-10922 (共 0 个槽位) d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5460 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 10923-16383 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.9:30435) 负责 0 个槽位: 5461-10922 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 5s job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 10s job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 15s job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 20s job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 26s job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 31s job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 36s job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 41s job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 46s job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 51s job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 56s job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 62s job pod status:kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs 1/1 Running 0 67s kb-post-provision-job-rediscl-vmwsrc-shard-w8k Success patch rediscl-vmwsrc shards 3 `kubectl patch cluster rediscl-vmwsrc --namespace ns-pwglh --type json -p '[***"op": "replace", "path": "/spec/shardingSpecs/0/shards", "value": '3'***]'` cluster.apps.kubeblocks.io/rediscl-vmwsrc patched check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Running Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-w8k-0 ns-pwglh rediscl-vmwsrc shard-w8k Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:27 UTC+0800 rediscl-vmwsrc-shard-w8k-1 ns-pwglh rediscl-vmwsrc shard-w8k Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:28 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0 rediscl-vmwsrc-shard-w8k-0;secondary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 rediscl-vmwsrc-shard-w8k-1 analyze cluster health check cluster connect `` ========== 节点: kb-post-provision-job-rediscl-vmwsrc-shard-w8k-596qs ========== 无法识别分片编号，跳过该节点 ========== 节点: kb-pre-terminate-job-rediscl-vmwsrc-shard-w8k-hql9k ========== 无法识别分片编号，跳过该节点 ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701028000 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701028679 22 connected 17a639cdebab9937570dac74b0d6911061d85e80 10.224.0.9:30599@30986,rediscl-vmwsrc-shard-w8k-1.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local slave b17f89b2dc34af8b72dc942eae08d48f9552450c 0 1756701028000 21 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701027518 16 connected 12288-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 14 connected 6827-10922 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756701028000 14 connected b17f89b2dc34af8b72dc942eae08d48f9552450c 10.224.0.9:30230@30631,rediscl-vmwsrc-shard-w8k-0.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local master - 0 1756701028000 21 connected 108-1364 5461-6826 10923-12287 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701028568 22 connected 0-107 1365-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:22 cluster_my_epoch:14 cluster_stats_messages_ping_sent:474 cluster_stats_messages_pong_sent:442 cluster_stats_messages_meet_sent:2 cluster_stats_messages_update_sent:28 cluster_stats_messages_sent:946 cluster_stats_messages_ping_received:442 cluster_stats_messages_pong_received:4719 cluster_stats_messages_update_received:9 cluster_stats_messages_received:5170 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 0 14 connected 17a639cdebab9937570dac74b0d6911061d85e80 10.224.0.9:30599@30986,rediscl-vmwsrc-shard-w8k-1.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local slave b17f89b2dc34af8b72dc942eae08d48f9552450c 0 1756701029000 21 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701029000 16 connected 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701029558 22 connected 0-269 1365-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701029656 16 connected b17f89b2dc34af8b72dc942eae08d48f9552450c 10.224.0.9:30230@30631,rediscl-vmwsrc-shard-w8k-0.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local master - 0 1756701029558 21 connected 270-1364 5461-6826 10923-12287 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701029000 14 connected 6827-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701028000 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:22 cluster_my_epoch:14 cluster_stats_messages_ping_sent:691 cluster_stats_messages_pong_sent:677 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_update_sent:8 cluster_stats_messages_sent:1392 cluster_stats_messages_ping_received:677 cluster_stats_messages_pong_received:5095 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_update_received:17 cluster_stats_messages_received:5793 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701029507 14 connected 6827-10922 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 22 connected 0-370 1365-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756701030379 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701029305 16 connected 12288-16383 17a639cdebab9937570dac74b0d6911061d85e80 10.224.0.9:30599@30986,rediscl-vmwsrc-shard-w8k-1.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local slave b17f89b2dc34af8b72dc942eae08d48f9552450c 0 1756701029000 21 connected b17f89b2dc34af8b72dc942eae08d48f9552450c 10.224.0.9:30230@30631,rediscl-vmwsrc-shard-w8k-0.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local master - 0 1756701029607 21 connected 371-1364 5461-6826 10923-12287 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701030000 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701030557 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:22 cluster_my_epoch:22 cluster_stats_messages_ping_sent:655 cluster_stats_messages_pong_sent:3387 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:12 cluster_stats_messages_sent:4061 cluster_stats_messages_ping_received:633 cluster_stats_messages_pong_received:4750 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:12 cluster_stats_messages_received:5401 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701032000 14 connected 6827-10922 17a639cdebab9937570dac74b0d6911061d85e80 10.224.0.9:30599@30986,rediscl-vmwsrc-shard-w8k-1.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local slave b17f89b2dc34af8b72dc942eae08d48f9552450c 0 1756701031807 21 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756701030000 14 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701031573 16 connected 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701031000 22 connected 0-453 1365-5460 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701030531 16 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 22 connected b17f89b2dc34af8b72dc942eae08d48f9552450c 10.224.0.9:30230@30631,rediscl-vmwsrc-shard-w8k-0.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local master - 0 1756701031000 21 connected 454-1364 5461-6826 10923-12287 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:22 cluster_my_epoch:22 cluster_stats_messages_ping_sent:683 cluster_stats_messages_pong_sent:717 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:6 cluster_stats_messages_sent:1422 cluster_stats_messages_ping_received:717 cluster_stats_messages_pong_received:5252 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:32 cluster_stats_messages_received:6006 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b17f89b2dc34af8b72dc942eae08d48f9552450c 10.224.0.9:30230@30631,rediscl-vmwsrc-shard-w8k-0.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local master - 0 1756701031220 21 connected 531-1364 5461-6826 10923-12287 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701032532 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756701032000 14 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701032000 22 connected 0-530 1365-5460 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 16 connected 12288-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701032244 22 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701032041 14 connected 6827-10922 17a639cdebab9937570dac74b0d6911061d85e80 10.224.0.9:30599@30986,rediscl-vmwsrc-shard-w8k-1.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local slave b17f89b2dc34af8b72dc942eae08d48f9552450c 0 1756701033262 21 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:22 cluster_my_epoch:16 cluster_stats_messages_ping_sent:674 cluster_stats_messages_pong_sent:631 cluster_stats_messages_meet_sent:2 cluster_stats_messages_update_sent:21 cluster_stats_messages_sent:1328 cluster_stats_messages_ping_received:627 cluster_stats_messages_pong_received:5325 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:1 cluster_stats_messages_update_received:14 cluster_stats_messages_received:5971 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 17a639cdebab9937570dac74b0d6911061d85e80 10.224.0.9:30599@30986,rediscl-vmwsrc-shard-w8k-1.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local slave b17f89b2dc34af8b72dc942eae08d48f9552450c 0 1756701033000 21 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701033812 14 connected 6827-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701034553 22 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 16 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756701032531 14 connected b17f89b2dc34af8b72dc942eae08d48f9552450c 10.224.0.9:30230@30631,rediscl-vmwsrc-shard-w8k-0.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local master - 0 1756701033000 21 connected 617-1364 5461-6826 10923-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701034000 16 connected 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701034000 22 connected 0-616 1365-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:22 cluster_my_epoch:16 cluster_stats_messages_ping_sent:688 cluster_stats_messages_pong_sent:697 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:8 cluster_stats_messages_sent:1409 cluster_stats_messages_ping_received:697 cluster_stats_messages_pong_received:5421 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_update_received:31 cluster_stats_messages_received:6154 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-w8k-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-w8k, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-w8k-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 17a639cdebab9937570dac74b0d6911061d85e80 10.224.0.9:30599@30986,rediscl-vmwsrc-shard-w8k-1.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local slave b17f89b2dc34af8b72dc942eae08d48f9552450c 0 1756701035019 21 connected b17f89b2dc34af8b72dc942eae08d48f9552450c 10.224.0.9:30230@30631,rediscl-vmwsrc-shard-w8k-0.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 21 connected 705-1364 5461-6826 10923-12287 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701034000 22 connected 0-704 1365-5460 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701034605 22 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701034000 16 connected 12288-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701034000 16 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701034621 14 connected 6827-10922 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756701035000 14 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:22 cluster_my_epoch:21 cluster_stats_messages_ping_sent:169 cluster_stats_messages_pong_sent:28852 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:13 cluster_stats_messages_sent:29035 cluster_stats_messages_ping_received:179 cluster_stats_messages_pong_received:908 cluster_stats_messages_meet_received:1 cluster_stats_messages_update_received:4 cluster_stats_messages_received:1092 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-w8k-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-w8k, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-w8k-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701036563 22 connected 0-803 1365-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756701036000 14 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701034626 14 connected 6827-10922 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701036564 22 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701036785 16 connected b17f89b2dc34af8b72dc942eae08d48f9552450c 10.224.0.9:30230@30631,rediscl-vmwsrc-shard-w8k-0.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local master - 0 1756701036000 21 connected 804-1364 5461-6826 10923-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701036000 16 connected 12288-16383 17a639cdebab9937570dac74b0d6911061d85e80 10.224.0.9:30599@30986,rediscl-vmwsrc-shard-w8k-1.rediscl-vmwsrc-shard-w8k-headless.ns-pwglh.svc.cluster.local myself,slave b17f89b2dc34af8b72dc942eae08d48f9552450c 0 0 21 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:22 cluster_my_epoch:21 cluster_stats_messages_ping_sent:137 cluster_stats_messages_pong_sent:137 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:34 cluster_stats_messages_sent:309 cluster_stats_messages_ping_received:137 cluster_stats_messages_pong_received:5064 cluster_stats_messages_update_received:28 cluster_stats_messages_received:5229 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g - w8k 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 17a639cdebab9937570dac74b0d6911061d85e80 10.224.0.9:30599 从节点 b17f89b2dc34af8b72dc942eae08d48f9552450c 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 12288-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 主节点 - 6827-10922 (共 0 个槽位) d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 从节点 fd7f24881ac1f6d15b989a864ebb711e66810506 b17f89b2dc34af8b72dc942eae08d48f9552450c 10.224.0.9:30230 主节点 - 108-1364 5461-6826 10923-12287 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-107 1365-5460 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-w8k-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-w8k-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 12288-16383 节点 fd7f24881ac1f6d15b989a864ebb711e66810506 (10.224.0.9:30435) 负责 0 个槽位: 6827-10922 节点 b17f89b2dc34af8b72dc942eae08d48f9552450c (10.224.0.9:30230) 负责 0 个槽位: 108-1364 5461-6826 10923-12287 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-107 1365-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 分片 w8k: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 4 个主节点和 4 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节��看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. job pod status: No resources found in ns-pwglh namespace. check rediscl-vmwsrc pre-terminate skip get shard component name set shard component name:shard-7wk test failover networklossover check cluster status before cluster-failover-networklossover check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-rediscl-vmwsrc --namespace ns-pwglh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-rediscl-vmwsrc" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-rediscl-vmwsrc" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-rediscl-vmwsrc namespace: ns-pwglh spec: selector: namespaces: - ns-pwglh labelSelectors: apps.kubeblocks.io/pod-name: rediscl-vmwsrc-shard-4wn-0 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-rediscl-vmwsrc.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-rediscl-vmwsrc created apply test-chaos-mesh-networklossover-rediscl-vmwsrc.yaml Success `rm -rf test-chaos-mesh-networklossover-rediscl-vmwsrc.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Updating Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701311559 23 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701311458 22 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701312565 23 connected 5462-6826 12288-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 25 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701312464 25 connected 6827-12287 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701312565 22 connected 0-5461 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:25 cluster_stats_messages_ping_sent:786 cluster_stats_messages_pong_sent:10314 cluster_stats_messages_meet_sent:2 cluster_stats_messages_update_sent:70 cluster_stats_messages_sent:11172 cluster_stats_messages_ping_received:759 cluster_stats_messages_pong_received:7610 cluster_stats_messages_update_received:12 cluster_stats_messages_received:8381 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701313000 23 connected 5462-6826 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701314556 22 connected 0-5461 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701313550 23 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701313000 25 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701313550 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:25 cluster_stats_messages_ping_sent:1174 cluster_stats_messages_pong_sent:1180 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:30 cluster_stats_messages_sent:2405 cluster_stats_messages_ping_received:1175 cluster_stats_messages_pong_received:9360 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_update_received:17 cluster_stats_messages_received:10560 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701313549 25 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 22 connected 0-5461 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701315562 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701315562 23 connected 5462-6826 12288-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701314556 23 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701315562 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:22 cluster_stats_messages_ping_sent:1157 cluster_stats_messages_pong_sent:10707 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:39 cluster_stats_messages_sent:11916 cluster_stats_messages_ping_received:1142 cluster_stats_messages_pong_received:7982 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_update_received:12 cluster_stats_messages_received:9144 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701316000 25 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701316669 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701315562 23 connected 5462-6826 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701316568 22 connected 0-5461 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701314657 23 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:22 cluster_stats_messages_ping_sent:1166 cluster_stats_messages_pong_sent:1202 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:31 cluster_stats_messages_sent:2415 cluster_stats_messages_ping_received:1202 cluster_stats_messages_pong_received:9353 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_update_received:32 cluster_stats_messages_received:10595 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701317000 23 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701317478 25 connected 6827-12287 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701316975 22 connected 0-5461 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 23 connected 5462-6826 12288-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701316000 22 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701317000 25 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:23 cluster_stats_messages_ping_sent:1203 cluster_stats_messages_pong_sent:10707 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:52 cluster_stats_messages_sent:11970 cluster_stats_messages_ping_received:1148 cluster_stats_messages_pong_received:8028 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:14 cluster_stats_messages_received:9197 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701317000 25 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701318487 22 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 23 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701317000 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701317482 23 connected 5462-6826 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701316979 22 connected 0-5461 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:23 cluster_stats_messages_ping_sent:1195 cluster_stats_messages_pong_sent:1188 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:32 cluster_stats_messages_sent:2431 cluster_stats_messages_ping_received:1188 cluster_stats_messages_pong_received:9382 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_update_received:31 cluster_stats_messages_received:10609 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 5462-6826 12288-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点 - 6827-12287 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5461 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 5462-6826 12288-16383 节�� d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 6827-12287 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5461 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-rediscl-vmwsrc --namespace ns-pwglh ` networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-rediscl-vmwsrc" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-rediscl-vmwsrc" not found check failover pod name failover pod name:rediscl-vmwsrc-shard-4wn-1 failover networklossover Success cluster update terminationPolicy WipeOut `kbcli cluster update rediscl-vmwsrc --termination-policy=WipeOut --namespace ns-pwglh ` cluster.apps.kubeblocks.io/rediscl-vmwsrc updated (no change) check cluster status `kbcli cluster list rediscl-vmwsrc --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc ns-pwglh WipeOut Running Sep 01,2025 11:17 UTC+0800 app.kubernetes.io/instance=rediscl-vmwsrc check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-shard-4wn-0 ns-pwglh rediscl-vmwsrc shard-4wn Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-4wn-1 ns-pwglh rediscl-vmwsrc shard-4wn Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-7wk-0 ns-pwglh rediscl-vmwsrc shard-7wk Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-7wk-1 ns-pwglh rediscl-vmwsrc shard-7wk Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 rediscl-vmwsrc-shard-d9g-0 ns-pwglh rediscl-vmwsrc shard-d9g Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:24 UTC+0800 rediscl-vmwsrc-shard-d9g-1 ns-pwglh rediscl-vmwsrc shard-d9g Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:23 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-shard-4wn-1 rediscl-vmwsrc-shard-7wk-0 rediscl-vmwsrc-shard-d9g-0;secondary: rediscl-vmwsrc-shard-4wn-0 rediscl-vmwsrc-shard-7wk-1 rediscl-vmwsrc-shard-d9g-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701397000 23 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701398080 22 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701397070 23 connected 5462-6826 12288-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 25 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701397576 25 connected 6827-12287 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701397577 22 connected 0-5461 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:25 cluster_stats_messages_ping_sent:939 cluster_stats_messages_pong_sent:10466 cluster_stats_messages_meet_sent:2 cluster_stats_messages_update_sent:70 cluster_stats_messages_sent:11477 cluster_stats_messages_ping_received:911 cluster_stats_messages_pong_received:7763 cluster_stats_messages_update_received:12 cluster_stats_messages_received:8686 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701398178 23 connected 5462-6826 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701399186 22 connected 0-5461 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701397173 23 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701397000 25 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701398000 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:25 cluster_stats_messages_ping_sent:1337 cluster_stats_messages_pong_sent:1339 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:30 cluster_stats_messages_sent:2727 cluster_stats_messages_ping_received:1334 cluster_stats_messages_pong_received:9523 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_update_received:17 cluster_stats_messages_received:10882 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701399000 25 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 22 connected 0-5461 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701398000 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701400196 23 connected 5462-6826 12288-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701399000 23 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701399189 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:22 cluster_stats_messages_ping_sent:1309 cluster_stats_messages_pong_sent:10872 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:39 cluster_stats_messages_sent:12233 cluster_stats_messages_ping_received:1307 cluster_stats_messages_pong_received:8134 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_update_received:12 cluster_stats_messages_received:9461 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701400801 25 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701400297 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701401305 23 connected 5462-6826 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701399290 22 connected 0-5461 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701401000 23 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:22 cluster_stats_messages_ping_sent:1312 cluster_stats_messages_pong_sent:1358 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:31 cluster_stats_messages_sent:2717 cluster_stats_messages_ping_received:1358 cluster_stats_messages_pong_received:9499 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_update_received:32 cluster_stats_messages_received:10897 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701402110 23 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701402000 25 connected 6827-12287 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701402110 22 connected 0-5461 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 23 connected 5462-6826 12288-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701401104 22 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701402513 25 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:23 cluster_stats_messages_ping_sent:1360 cluster_stats_messages_pong_sent:10857 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:52 cluster_stats_messages_sent:12277 cluster_stats_messages_ping_received:1298 cluster_stats_messages_pong_received:8185 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:14 cluster_stats_messages_received:9504 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701402613 25 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701403117 22 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 23 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701402109 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701403621 23 connected 5462-6826 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701402000 22 connected 0-5461 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:23 cluster_stats_messages_ping_sent:1359 cluster_stats_messages_pong_sent:1341 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:32 cluster_stats_messages_sent:2748 cluster_stats_messages_ping_received:1341 cluster_stats_messages_pong_received:9546 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_update_received:31 cluster_stats_messages_received:10926 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 5462-6826 12288-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点 - 6827-12287 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5461 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 5462-6826 12288-16383 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 6827-12287 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5461 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl get backupschedule -l app.kubernetes.io/instance=rediscl-vmwsrc ` `kubectl get backupschedule rediscl-vmwsrc-shard-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/rediscl-vmwsrc-shard-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/rediscl-vmwsrc-shard-backup-schedule patched check backup status `kbcli cluster list-backups rediscl-vmwsrc --namespace ns-pwglh ` No backups found in ns-pwglh namespace. No backups found in ns-pwglh namespace. backup_status:-- No backups found in ns-pwglh namespace. backup_status:-- No backups found in ns-pwglh namespace. backup_status:-- No backups found in ns-pwglh namespace. backup_status:-- `kubectl get backupschedule -l app.kubernetes.io/instance=rediscl-vmwsrc ` `kubectl get backupschedule rediscl-vmwsrc-shard-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/rediscl-vmwsrc-shard-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/rediscl-vmwsrc-shard-backup-schedule patched backup_status:rediscl-vmwsrc-datafile-Running backup_status:rediscl-vmwsrc-datafile-Running backup_status:rediscl-vmwsrc-datafile-Running check backup status done backup_status:rediscl-vmwsrc-datafile-20250901043800 ns-pwglh rediscl-vmwsrc datafile Completed 7479 19s Sep 01,2025 12:38 UTC+0800 Sep 01,2025 12:38 UTC+0800 Sep 08,2025 12:38 UTC+0800 `kubectl get backupschedule -l app.kubernetes.io/instance=rediscl-vmwsrc ` `kubectl get backupschedule rediscl-vmwsrc-shard-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/rediscl-vmwsrc-shard-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/rediscl-vmwsrc-shard-backup-schedule patched (no change) cluster restore backup Error from server (NotFound): opsrequests.apps.kubeblocks.io "rediscl-vmwsrc-backup" not found `kbcli cluster describe-backup rediscl-vmwsrc-datafile-20250901043800 --namespace ns-pwglh ` Name: rediscl-vmwsrc-datafile-20250901043800 Cluster: rediscl-vmwsrc Namespace: ns-pwglh Spec: Method: datafile Policy Name: rediscl-vmwsrc-shard-backup-policy Status: Phase: Completed Total Size: 7479 ActionSet Name: redis-cluster-backup Repository: backuprepo-kbcli-test Duration: 19s Expiration Time: Sep 08,2025 12:38 UTC+0800 Start Time: Sep 01,2025 12:38 UTC+0800 Completion Time: Sep 01,2025 12:38 UTC+0800 Path: /ns-pwglh/rediscl-vmwsrc-3879c217-d4dc-4537-84da-8bfafffde0da/shard/rediscl-vmwsrc-datafile-20250901043800 Warning Events: `kbcli cluster restore rediscl-vmwsrc-backup --backup rediscl-vmwsrc-datafile-20250901043800 --namespace ns-pwglh ` Cluster rediscl-vmwsrc-backup created check cluster status `kbcli cluster list rediscl-vmwsrc-backup --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc-backup ns-pwglh WipeOut Sep 01,2025 12:38 UTC+0800 cluster_status: cluster_status: cluster_status: cluster_status: cluster_status: cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc-backup --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-backup-shard-qdg-0 ns-pwglh rediscl-vmwsrc-backup shard-qdg Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:38 UTC+0800 rediscl-vmwsrc-backup-shard-qdg-1 ns-pwglh rediscl-vmwsrc-backup shard-qdg Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:39 UTC+0800 rediscl-vmwsrc-backup-shard-rwn-0 ns-pwglh rediscl-vmwsrc-backup shard-rwn Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:38 UTC+0800 rediscl-vmwsrc-backup-shard-rwn-1 ns-pwglh rediscl-vmwsrc-backup shard-rwn Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:39 UTC+0800 rediscl-vmwsrc-backup-shard-w4z-0 ns-pwglh rediscl-vmwsrc-backup shard-w4z Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:38 UTC+0800 rediscl-vmwsrc-backup-shard-w4z-1 ns-pwglh rediscl-vmwsrc-backup shard-w4z Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:39 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-backup-shard-qdg-0 rediscl-vmwsrc-backup-shard-qdg-1 rediscl-vmwsrc-backup-shard-rwn-0 rediscl-vmwsrc-backup-shard-rwn-1 rediscl-vmwsrc-backup-shard-w4z-0 rediscl-vmwsrc-backup-shard-w4z-1;secondary: rediscl-vmwsrc-backup-shard-qdg-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701600000 23 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701600000 22 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701600542 23 connected 5462-6826 12288-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 25 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701601550 25 connected 6827-12287 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701600039 22 connected 0-5461 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:25 cluster_stats_messages_ping_sent:1306 cluster_stats_messages_pong_sent:10831 cluster_stats_messages_meet_sent:2 cluster_stats_messages_update_sent:70 cluster_stats_messages_sent:12209 cluster_stats_messages_ping_received:1276 cluster_stats_messages_pong_received:8130 cluster_stats_messages_update_received:12 cluster_stats_messages_received:9418 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701602560 23 connected 5462-6826 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701602661 22 connected 0-5461 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701602000 23 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701600647 25 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701601554 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:25 cluster_stats_messages_ping_sent:1685 cluster_stats_messages_pong_sent:1693 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:30 cluster_stats_messages_sent:3429 cluster_stats_messages_ping_received:1688 cluster_stats_messages_pong_received:9871 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_update_received:17 cluster_stats_messages_received:11584 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701603000 25 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 22 connected 0-5461 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701603000 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701603000 23 connected 5462-6826 12288-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701603663 23 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701603563 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:22 cluster_stats_messages_ping_sent:1663 cluster_stats_messages_pong_sent:11224 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:39 cluster_stats_messages_sent:12939 cluster_stats_messages_ping_received:1659 cluster_stats_messages_pong_received:8488 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_update_received:12 cluster_stats_messages_received:10167 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701603361 25 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701603000 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701604000 23 connected 5462-6826 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701603863 22 connected 0-5461 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701604870 23 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:22 cluster_stats_messages_ping_sent:1668 cluster_stats_messages_pong_sent:1717 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:31 cluster_stats_messages_sent:3432 cluster_stats_messages_ping_received:1717 cluster_stats_messages_pong_received:9855 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_update_received:32 cluster_stats_messages_received:11612 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701605597 23 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701605000 25 connected 6827-12287 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701604591 22 connected 0-5461 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 23 connected 5462-6826 12288-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701606101 22 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701605597 25 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:23 cluster_stats_messages_ping_sent:1731 cluster_stats_messages_pong_sent:11215 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:52 cluster_stats_messages_sent:13006 cluster_stats_messages_ping_received:1656 cluster_stats_messages_pong_received:8556 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:14 cluster_stats_messages_received:10233 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701605580 25 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701606000 22 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 23 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701607000 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701606686 23 connected 5462-6826 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701606585 22 connected 0-5461 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:23 cluster_stats_messages_ping_sent:1715 cluster_stats_messages_pong_sent:1708 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:32 cluster_stats_messages_sent:3471 cluster_stats_messages_ping_received:1708 cluster_stats_messages_pong_received:9902 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_update_received:31 cluster_stats_messages_received:11649 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 5462-6826 12288-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点 - 6827-12287 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5461 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在��析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 5462-6826 12288-16383 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 6827-12287 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5461 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kbcli cluster describe-backup rediscl-vmwsrc-datafile-20250901043800 --namespace ns-pwglh ` Name: rediscl-vmwsrc-datafile-20250901043800 Cluster: rediscl-vmwsrc Namespace: ns-pwglh Spec: Method: datafile Policy Name: rediscl-vmwsrc-shard-backup-policy Status: Phase: Completed Total Size: 7479 ActionSet Name: redis-cluster-backup Repository: backuprepo-kbcli-test Duration: 19s Expiration Time: Sep 08,2025 12:38 UTC+0800 Start Time: Sep 01,2025 12:38 UTC+0800 Completion Time: Sep 01,2025 12:38 UTC+0800 Path: /ns-pwglh/rediscl-vmwsrc-3879c217-d4dc-4537-84da-8bfafffde0da/shard/rediscl-vmwsrc-datafile-20250901043800 Warning Events: cluster connect kbcli unsupported engine type: shard-7wk delete cluster rediscl-vmwsrc-backup `kbcli cluster delete rediscl-vmwsrc-backup --auto-approve --namespace ns-pwglh ` Cluster rediscl-vmwsrc-backup deleted pod_info:rediscl-vmwsrc-backup-shard-qdg-0 3/3 Running 0 2m22s rediscl-vmwsrc-backup-shard-qdg-1 3/3 Running 0 115s rediscl-vmwsrc-backup-shard-rwn-0 3/3 Running 0 2m21s rediscl-vmwsrc-backup-shard-rwn-1 3/3 Running 0 115s rediscl-vmwsrc-backup-shard-w4z-0 3/3 Running 0 2m20s rediscl-vmwsrc-backup-shard-w4z-1 3/3 Running 0 115s No resources found in ns-pwglh namespace. delete cluster pod done No resources found in ns-pwglh namespace. check cluster resource non-exist OK: pvc No resources found in ns-pwglh namespace. delete cluster done No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups rediscl-vmwsrc-datafile-20250901043800 --namespace ns-pwglh ` backup.dataprotection.kubeblocks.io/rediscl-vmwsrc-datafile-20250901043800 patched `kbcli cluster delete-backup rediscl-vmwsrc --name rediscl-vmwsrc-datafile-20250901043800 --force --auto-approve --namespace ns-pwglh ` Backup rediscl-vmwsrc-datafile-20250901043800 deleted No opsrequests found in ns-pwglh namespace. cluster datafile backup `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.name***"` `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.namespace***"` `kubectl get secrets kb-backuprepo-lkjhb -n kb-luwux -o jsonpath="***.data.accessKeyId***"` `kubectl get secrets kb-backuprepo-lkjhb -n kb-luwux -o jsonpath="***.data.secretAccessKey***"` KUBEBLOCKS NAMESPACE:kb-luwux get kubeblocks namespace done `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-luwux -o jsonpath="***.items[0].data.root-user***"` `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-luwux -o jsonpath="***.items[0].data.root-password***"` minio_user:kbclitest,minio_password:kbclitest,minio_endpoint:kbcli-test-minio.kb-luwux.svc.cluster.local:9000 list minio bucket kbcli-test `echo 'mc config host add minioserver http://kbcli-test-minio.kb-luwux.svc.cluster.local:9000 kbclitest kbclitest;mc ls minioserver' | kubectl exec -it kbcli-test-minio-95fd7c555-gl88t --namespace kb-luwux -- bash` Unable to use a TTY - input is not a terminal or the right kind of file list minio bucket done default backuprepo:backuprepo-kbcli-test exists `kbcli cluster backup rediscl-vmwsrc --method datafile --namespace ns-pwglh ` Backup backup-ns-pwglh-rediscl-vmwsrc-20250901124134 created successfully, you can view the progress: kbcli cluster list-backups --name=backup-ns-pwglh-rediscl-vmwsrc-20250901124134 -n ns-pwglh check backup status `kbcli cluster list-backups rediscl-vmwsrc --namespace ns-pwglh ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION CREATE-TIME COMPLETION-TIME EXPIRATION backup-ns-pwglh-rediscl-vmwsrc-20250901124134 ns-pwglh rediscl-vmwsrc datafile Sep 01,2025 12:41 UTC+0800 backup_status:rediscl-vmwsrc-datafile-Running backup_status:rediscl-vmwsrc-datafile-Running backup_status:rediscl-vmwsrc-datafile-Running check backup status done backup_status:backup-ns-pwglh-rediscl-vmwsrc-20250901124134 ns-pwglh rediscl-vmwsrc datafile Completed 7481 10s Sep 01,2025 12:41 UTC+0800 Sep 01,2025 12:41 UTC+0800 cluster restore backup Error from server (NotFound): opsrequests.apps.kubeblocks.io "rediscl-vmwsrc-backup" not found `kbcli cluster describe-backup backup-ns-pwglh-rediscl-vmwsrc-20250901124134 --namespace ns-pwglh ` Name: backup-ns-pwglh-rediscl-vmwsrc-20250901124134 Cluster: rediscl-vmwsrc Namespace: ns-pwglh Spec: Method: datafile Policy Name: rediscl-vmwsrc-shard-backup-policy Status: Phase: Completed Total Size: 7481 ActionSet Name: redis-cluster-backup Repository: backuprepo-kbcli-test Duration: 10s Start Time: Sep 01,2025 12:41 UTC+0800 Completion Time: Sep 01,2025 12:41 UTC+0800 Path: /ns-pwglh/rediscl-vmwsrc-3879c217-d4dc-4537-84da-8bfafffde0da/shard/backup-ns-pwglh-rediscl-vmwsrc-20250901124134 Warning Events: `kbcli cluster restore rediscl-vmwsrc-backup --backup backup-ns-pwglh-rediscl-vmwsrc-20250901124134 --namespace ns-pwglh ` Cluster rediscl-vmwsrc-backup created check cluster status `kbcli cluster list rediscl-vmwsrc-backup --show-labels --namespace ns-pwglh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-vmwsrc-backup ns-pwglh WipeOut Sep 01,2025 12:41 UTC+0800 cluster_status: cluster_status: cluster_status: cluster_status: cluster_status: cluster_status: cluster_status: cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-vmwsrc-backup --namespace ns-pwglh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-vmwsrc-backup-shard-btv-0 ns-pwglh rediscl-vmwsrc-backup shard-btv Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:42 UTC+0800 rediscl-vmwsrc-backup-shard-btv-1 ns-pwglh rediscl-vmwsrc-backup shard-btv Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:43 UTC+0800 rediscl-vmwsrc-backup-shard-dpq-0 ns-pwglh rediscl-vmwsrc-backup shard-dpq Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:42 UTC+0800 rediscl-vmwsrc-backup-shard-dpq-1 ns-pwglh rediscl-vmwsrc-backup shard-dpq Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:42 UTC+0800 rediscl-vmwsrc-backup-shard-rbr-0 ns-pwglh rediscl-vmwsrc-backup shard-rbr Running primary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:42 UTC+0800 rediscl-vmwsrc-backup-shard-rbr-1 ns-pwglh rediscl-vmwsrc-backup shard-rbr Running secondary 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 12:42 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-vmwsrc-backup-shard-btv-0 rediscl-vmwsrc-backup-shard-dpq-0 rediscl-vmwsrc-backup-shard-rbr-0;secondary: rediscl-vmwsrc-backup-shard-btv-1 rediscl-vmwsrc-backup-shard-dpq-1 rediscl-vmwsrc-backup-shard-rbr-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-vmwsrc-shard-4wn-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701843000 23 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701844553 22 connected 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701844251 23 connected 5462-6826 12288-16383 fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,slave d4512197faed320ad44935144bd1e67f8e4e1925 0 0 25 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701843244 25 connected 6827-12287 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701844755 22 connected 0-5461 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:25 cluster_stats_messages_ping_sent:1756 cluster_stats_messages_pong_sent:11257 cluster_stats_messages_meet_sent:2 cluster_stats_messages_update_sent:70 cluster_stats_messages_sent:13085 cluster_stats_messages_ping_received:1702 cluster_stats_messages_pong_received:8580 cluster_stats_messages_update_received:12 cluster_stats_messages_received:10294 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-4wn-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-4wn, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-4wn-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701844353 23 connected 5462-6826 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701844000 22 connected 0-5461 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701845359 23 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701844000 25 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701845000 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:25 cluster_stats_messages_ping_sent:2111 cluster_stats_messages_pong_sent:2115 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:30 cluster_stats_messages_sent:4277 cluster_stats_messages_ping_received:2110 cluster_stats_messages_pong_received:10297 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_update_received:17 cluster_stats_messages_received:12432 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701846000 25 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 22 connected 0-5461 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701847428 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701846522 23 connected 5462-6826 12288-16383 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701846421 23 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701845000 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:22 cluster_stats_messages_ping_sent:2089 cluster_stats_messages_pong_sent:11667 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:39 cluster_stats_messages_sent:13808 cluster_stats_messages_ping_received:2102 cluster_stats_messages_pong_received:8914 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_update_received:12 cluster_stats_messages_received:11036 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-7wk-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-7wk, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-7wk-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701848584 25 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701847578 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701848000 23 connected 5462-6826 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701847000 22 connected 0-5461 f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701848584 23 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local myself,slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 0 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:22 cluster_stats_messages_ping_sent:2081 cluster_stats_messages_pong_sent:2149 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:31 cluster_stats_messages_sent:4277 cluster_stats_messages_ping_received:2149 cluster_stats_messages_pong_received:10268 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_update_received:32 cluster_stats_messages_received:12457 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-0 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 0 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 1756701848336 23 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701847329 25 connected 6827-12287 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701847530 22 connected 0-5461 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 23 connected 5462-6826 12288-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701849343 22 connected fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701848000 25 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:23 cluster_stats_messages_ping_sent:2176 cluster_stats_messages_pong_sent:11654 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:52 cluster_stats_messages_sent:13890 cluster_stats_messages_ping_received:2095 cluster_stats_messages_pong_received:9001 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:14 cluster_stats_messages_received:11117 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-vmwsrc-shard-d9g-1 ========== 检测到基础名称: rediscl-vmwsrc-shard-d9g, 分片编号: 1 使用 Secret: rediscl-vmwsrc-shard-d9g-account-default 密码获取成功，前缀: 55... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave d4512197faed320ad44935144bd1e67f8e4e1925 0 1756701850048 25 connected b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756701850000 22 connected f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177@31232,rediscl-vmwsrc-shard-d9g-1.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local myself,slave 4c0a70fc5f0991c13eca381ded257b22beea970b 0 0 23 connected d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local master - 0 1756701850000 25 connected 6827-12287 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master - 0 1756701849000 23 connected 5462-6826 12288-16383 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756701850452 22 connected 0-5461 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:25 cluster_my_epoch:23 cluster_stats_messages_ping_sent:2150 cluster_stats_messages_pong_sent:2142 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:32 cluster_stats_messages_sent:4340 cluster_stats_messages_ping_received:2142 cluster_stats_messages_pong_received:10337 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_update_received:31 cluster_stats_messages_received:12518 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-vmwsrc-shard-4wn-0 作为参考基准 检测到以下分片标识符: - 4wn - 7wk - d9g 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- f49b9ec6a80833cb30167969e9f74558c9513f47 10.224.0.9:32177 从节点 4c0a70fc5f0991c13eca381ded257b22beea970b b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579 从节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167 主节点 - 5462-6826 12288-16383 (共 0 个槽位) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435 从节点 d4512197faed320ad44935144bd1e67f8e4e1925 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489 主节点 - 6827-12287 (共 0 个槽位) f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971 主节点 - 0-5461 (共 0 个槽位) 正在分析节点 rediscl-vmwsrc-shard-4wn-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-7wk-1 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-0 的拓扑视图... 正在分析节点 rediscl-vmwsrc-shard-d9g-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 4c0a70fc5f0991c13eca381ded257b22beea970b (10.224.0.9:32167) 负责 0 个槽位: 5462-6826 12288-16383 节点 d4512197faed320ad44935144bd1e67f8e4e1925 (10.224.0.9:32489) 负责 0 个槽位: 6827-12287 节点 f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 (10.224.0.9:30971) 负责 0 个槽位: 0-5461 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4wn: 1 个主节点, 1 个从节点 ✅ 分片 7wk: 1 个主节点, 1 个从节点 ✅ 分片 d9g: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kbcli cluster describe-backup backup-ns-pwglh-rediscl-vmwsrc-20250901124134 --namespace ns-pwglh ` Name: backup-ns-pwglh-rediscl-vmwsrc-20250901124134 Cluster: rediscl-vmwsrc Namespace: ns-pwglh Spec: Method: datafile Policy Name: rediscl-vmwsrc-shard-backup-policy Status: Phase: Completed Total Size: 7481 ActionSet Name: redis-cluster-backup Repository: backuprepo-kbcli-test Duration: 10s Start Time: Sep 01,2025 12:41 UTC+0800 Completion Time: Sep 01,2025 12:41 UTC+0800 Path: /ns-pwglh/rediscl-vmwsrc-3879c217-d4dc-4537-84da-8bfafffde0da/shard/backup-ns-pwglh-rediscl-vmwsrc-20250901124134 Warning Events: cluster connect kbcli unsupported engine type: shard-7wk delete cluster rediscl-vmwsrc-backup `kbcli cluster delete rediscl-vmwsrc-backup --auto-approve --namespace ns-pwglh ` Cluster rediscl-vmwsrc-backup deleted pod_info:rediscl-vmwsrc-backup-shard-btv-0 3/3 Running 0 2m47s rediscl-vmwsrc-backup-shard-btv-1 3/3 Running 0 113s rediscl-vmwsrc-backup-shard-dpq-0 3/3 Running 0 3m rediscl-vmwsrc-backup-shard-dpq-1 3/3 Running 0 2m33s rediscl-vmwsrc-backup-shard-rbr-0 3/3 Running 0 2m59s rediscl-vmwsrc-backup-shard-rbr-1 3/3 Running 0 2m34s No resources found in ns-pwglh namespace. delete cluster pod done No resources found in ns-pwglh namespace. check cluster resource non-exist OK: pvc No resources found in ns-pwglh namespace. delete cluster done No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups backup-ns-pwglh-rediscl-vmwsrc-20250901124134 --namespace ns-pwglh ` backup.dataprotection.kubeblocks.io/backup-ns-pwglh-rediscl-vmwsrc-20250901124134 patched `kbcli cluster delete-backup rediscl-vmwsrc --name backup-ns-pwglh-rediscl-vmwsrc-20250901124134 --force --auto-approve --namespace ns-pwglh ` Backup backup-ns-pwglh-rediscl-vmwsrc-20250901124134 deleted No opsrequests found in ns-pwglh namespace. cluster list-logs `kbcli cluster list-logs rediscl-vmwsrc --namespace ns-pwglh ` No log files found. You can enable the log feature with the kbcli command below. kbcli cluster update rediscl-vmwsrc --enable-all-logs=true --namespace ns-pwglh Error from server (NotFound): pods "rediscl-vmwsrc-shard-4wn-1" not found cluster logs `kbcli cluster logs rediscl-vmwsrc --tail 30 --namespace ns-pwglh ` Defaulted container "redis-cluster" out of: redis-cluster, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) fd7f24881ac1f6d15b989a864ebb711e66810506 10.224.0.9:30435@31249,rediscl-vmwsrc-shard-4wn-0.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local myself,master - 0 0 14 connected 5461-10922 4c0a70fc5f0991c13eca381ded257b22beea970b 10.224.0.9:32167@32475,rediscl-vmwsrc-shard-d9g-0.rediscl-vmwsrc-shard-d9g-headless.ns-pwglh.svc.cluster.local master,fail - 0 1756700698000 16 connected 10923-16383 b911daf71479a037983b183875a34325c6508850 10.224.0.9:31579@30671,rediscl-vmwsrc-shard-7wk-1.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local slave f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 0 1756700697566 1 connected f7fa9d9689dab4e672f36fb8abe1889dc2fc02e0 10.224.0.9:30971@31723,rediscl-vmwsrc-shard-7wk-0.rediscl-vmwsrc-shard-7wk-headless.ns-pwglh.svc.cluster.local master - 0 1756700697767 1 connected 0-5460 d4512197faed320ad44935144bd1e67f8e4e1925 10.224.0.9:32489@31201,rediscl-vmwsrc-shard-4wn-1.rediscl-vmwsrc-shard-4wn-headless.ns-pwglh.svc.cluster.local slave fd7f24881ac1f6d15b989a864ebb711e66810506 0 1756700698773 14 connected' +++ grep myself +++ awk '***print $2***' +++ awk -F : '***print $1***' ++ cluster_announce_ip=10.224.0.9 ++ echo 10.224.0.9 + current_announce_ip=10.224.0.9 target: 10.224.0.9:30435, current_announce_ip: 10.224.0.9 + echo 'target: 10.224.0.9:30435, current_announce_ip: 10.224.0.9' + '[' -z 10.224.0.9 ']' + set +x Meet command: redis-cli -h 10.224.0.9 -p 32167 -a ******** cluster meet 10.224.0.9 30435 31249 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. OK Meet the node 10.224.0.9:30435 successfully with new announce ip 10.224.0.9... + break + wait_random_second 10 1 + local max_time=10 + local min_time=1 Sleeping for 2 seconds + local random_time=2 + echo 'Sleeping for 2 seconds' + sleep 2 Node rediscl-vmwsrc-shard-d9g-0 is already in the cluster, skipping scale out replica... + echo 'Node rediscl-vmwsrc-shard-d9g-0 is already in the cluster, skipping scale out replica...' + exit 0 delete cluster rediscl-vmwsrc `kbcli cluster delete rediscl-vmwsrc --auto-approve --namespace ns-pwglh ` Cluster rediscl-vmwsrc deleted pod_info:rediscl-vmwsrc-shard-4wn-0 3/3 Running 1 (19m ago) 21m rediscl-vmwsrc-shard-4wn-1 3/3 Running 0 21m rediscl-vmwsrc-shard-7wk-0 3/3 Running 0 21m rediscl-vmwsrc-shard-7wk-1 3/3 Running 0 21m rediscl-vmwsrc-shard-d9g-0 3/3 Running 0 21m rediscl-vmwsrc-shard-d9g-1 3/3 Running 0 21m No resources found in ns-pwglh namespace. delete cluster pod done No resources found in ns-pwglh namespace. check cluster resource non-exist OK: pvc No resources found in ns-pwglh namespace. delete cluster done No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. No resources found in ns-pwglh namespace. Redis Cluster Test Suite All Done! --------------------------------------Redis Cluster (Topology = redis-cluster Replicas 2) Test Result-------------------------------------- [PASSED]|[Create]|[ClusterDefinition=redis;ClusterVersion=redis-7.0.6;]|[Description=Create a cluster with the specified cluster definition redis and cluster version redis-7.0.6] [PASSED]|[HorizontalScaling Out]|[ComponentName=shard-4wn]|[Description=HorizontalScaling Out the cluster specify component shard-4wn] [PASSED]|[HorizontalScaling In]|[ComponentName=shard-4wn]|[Description=HorizontalScaling In the cluster specify component shard-4wn] [WARNING]|[CheckFailover]|[FailoverType=fullcpu]|[Description=-] [PASSED]|[Failover]|[HA=Full CPU;Durations=2m;ComponentName=shard-4wn]|[Description=Simulates conditions where pods experience CPU full either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high CPU load.] [PASSED]|[No-Failover]|[HA=Time Offset;Durations=2m;ComponentName=shard-4wn]|[Description=Simulates a time offset scenario thereby testing the application's resilience to potential slowness/unavailability of some replicas due to time offset.] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [PASSED]|[No-Failover]|[HA=Network Delay;Durations=2m;ComponentName=shard-4wn]|[Description=Simulates network delay fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to delay network.] [PASSED]|[Failover]|[HA=Pod Kill;ComponentName=shard-4wn]|[Description=Simulates conditions where pods experience kill for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to kill.] [PASSED]|[Bench]|[ComponentName=shard-4wn]|[Description=Bench the cluster service with shard-4wn component] [PASSED]|[Failover]|[HA=Delete Pod All;ComponentName=shard-4wn]|[Description=] [PASSED]|[VolumeExpansion]|[ComponentName=shard-4wn]|[Description=VolumeExpansion the cluster specify component shard-4wn] [PASSED]|[Failover]|[HA=Pod Failure;Durations=2m;ComponentName=shard-4wn]|[Description=Simulates conditions where pods experience failure for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to failure.] [PASSED]|[Stop]|[-]|[Description=Stop the cluster] [PASSED]|[Start]|[-]|[Description=Start the cluster] [PASSED]|[Failover]|[HA=Evicting Pod;ComponentName=shard-4wn]|[Description=Simulates conditions where pods evicting either due to node drained thereby testing the application's resilience to unavailability of some replicas due to evicting.] [PASSED]|[Failover]|[HA=Kill 1;ComponentName=shard-4wn]|[Description=Simulates conditions where process 1 killed either due to expected/undesired processes thereby testing the application's resilience to unavailability of some replicas due to abnormal termination signals.] [PASSED]|[Update]|[Logs=Enable]|[Description=Update the cluster Logs enable] [PASSED]|[No-Failover]|[HA=Network Duplicate;Durations=2m;ComponentName=shard-4wn]|[Description=Simulates network duplicate fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to duplicate network.] [PASSED]|[VerticalScaling]|[ComponentName=shard-4wn]|[Description=VerticalScaling the cluster specify component shard-4wn] [PASSED]|[Failover]|[HA=OOM;Durations=2m;ComponentName=shard-4wn]|[Description=Simulates conditions where pods experience OOM either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Memory load.] [PASSED]|[HorizontalScaling Out]|[ShardsName=shard]|[Description=HorizontalScaling Out the cluster] [SKIPPED]|[HorizontalScaling In]|[ShardsName=shard]|[Description=HorizontalScaling In the cluster] [PASSED]|[Failover]|[HA=Network Loss;Durations=2m;ComponentName=shard-7wk]|[Description=Simulates network loss fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to loss network.] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Backup]|[Schedule=true;BackupMethod=datafile]|[Description=The cluster Schedule datafile Backup] [PASSED]|[Restore]|[Schedule=true;BackupMethod=datafile]|[Description=The cluster Schedule datafile Restore] [PASSED]|[Delete Restore Cluster]|[Schedule=true;BackupMethod=datafile]|[Description=Delete the Schedule datafile restore cluster] [PASSED]|[Backup]|[BackupMethod=datafile]|[Description=The cluster datafile Backup] [PASSED]|[Restore]|[BackupMethod=datafile]|[Description=The cluster datafile Restore] [PASSED]|[Delete Restore Cluster]|[BackupMethod=datafile]|[Description=Delete the datafile restore cluster] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]