source commons files source engines files source kubeblocks files `kubectl get namespace | grep ns-hbjkr ` `kubectl create namespace ns-hbjkr` namespace/ns-hbjkr created create namespace ns-hbjkr done download kbcli `gh release list --repo apecloud/kbcli --limit 100 | (grep "1.0" || true)` `curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v1.0.0` Your system is linux_amd64 Installing kbcli ... Downloading ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 33.6M 100 33.6M 0 0 87.0M 0 --:--:-- --:--:-- --:--:-- 87.0M kbcli installed successfully. Kubernetes: v1.32.5-eks-5d4a308 KubeBlocks: 1.0.0 kbcli: 1.0.0 Make sure your docker service is running and begin your journey with kbcli: kbcli playground init For more information on how to get started, please visit: https://kubeblocks.io download kbcli v1.0.0 done Kubernetes: v1.32.5-eks-5d4a308 KubeBlocks: 1.0.0 kbcli: 1.0.0 Kubernetes Env: v1.32.5-eks-5d4a308 check snapshot controller check snapshot controller done eks default-vsc found POD_RESOURCES: No resources found found default storage class: gp3 KubeBlocks version is:1.0.0 skip upgrade KubeBlocks current KubeBlocks version: 1.0.0 Error: no repositories to show helm repo add chaos-mesh https://charts.chaos-mesh.org "chaos-mesh" has been added to your repositories add helm chart repo chaos-mesh success chaos mesh already installed check component definition set component name:redis set component version set component version:redis-cluster set service versions:7.2.7,7.2.4,7.0.6 set service versions sorted:7.0.6,7.2.4,7.2.7 set redis component definition set redis component definition redis-cluster-7-1.0.0-alpha.0 set replicas first:2,7.0.6|2,7.2.4|2,7.2.7 set replicas third:2,7.2.7 set replicas fourth:2,7.2.4 set minimum cmpv service version set minimum cmpv service version replicas:2,7.2.4 REPORT_COUNT:1 set redis component definition set redis component definition redis-cluster-7-1.0.0-alpha.0 LIMIT_CPU:0.1 LIMIT_MEMORY:0.5 storage size: 1 No resources found in ns-hbjkr namespace. termination_policy:WipeOut create 2 replica WipeOut redis cluster check component definition set component definition by component version check cmpd by labels set component definition1: redis-cluster-7-1.0.0-alpha.0 by component version:redis-cluster apiVersion: apps.kubeblocks.io/v1 kind: Cluster metadata: name: rediscl-epxpxf namespace: ns-hbjkr spec: terminationPolicy: WipeOut shardings: - name: shard shards: 3 template: name: redis componentDef: redis-cluster-7-1.0.0-alpha.0 replicas: 2 services: - name: redis-advertised serviceType: NodePort podService: true systemAccounts: - name: default passwordConfig: length: 10 numDigits: 5 numSymbols: 0 letterCase: MixedCases seed: redisc resources: limits: cpu: 100m memory: 0.5Gi requests: cpu: 100m memory: 0.5Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi `kubectl apply -f test_create_rediscl-epxpxf.yaml` cluster.apps.kubeblocks.io/rediscl-epxpxf created apply test_create_rediscl-epxpxf.yaml Success `rm -rf test_create_rediscl-epxpxf.yaml` check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Creating May 28,2025 11:38 UTC+0800 cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748403666593 2 connected 5461-10922 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748403667599 2 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748403665000 3 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748403667698 1 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748403666694 1 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748403666000 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:98 cluster_stats_messages_pong_sent:103 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:202 cluster_stats_messages_ping_received:102 cluster_stats_messages_pong_received:99 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:202 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748403672458 1 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748403671455 1 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748403672560 2 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748403671000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748403671553 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748403672000 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:90 cluster_stats_messages_pong_sent:94 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:185 cluster_stats_messages_ping_received:94 cluster_stats_messages_pong_received:91 cluster_stats_messages_received:185 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748403676000 1 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 1748403677999 1748403675456 2 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748403677466 2 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748403676000 1 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748403676059 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748403676461 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:123 cluster_stats_messages_pong_sent:130 cluster_stats_messages_sent:253 cluster_stats_messages_ping_received:127 cluster_stats_messages_pong_received:123 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:253 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748403682000 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748403682702 1 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748403681000 2 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748403681095 3 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748403680000 1 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748403681597 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:125 cluster_stats_messages_pong_sent:124 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:250 cluster_stats_messages_ping_received:124 cluster_stats_messages_pong_received:126 cluster_stats_messages_received:250 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748403686000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748403687000 1 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748403686788 1 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748403686587 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748403686000 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748403686587 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:139 cluster_stats_messages_pong_sent:140 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:280 cluster_stats_messages_ping_received:139 cluster_stats_messages_pong_received:140 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:280 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748403690608 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748403691000 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748403691612 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748403690000 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748403690000 1 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748403691511 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:144 cluster_stats_messages_pong_sent:133 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:278 cluster_stats_messages_ping_received:133 cluster_stats_messages_pong_received:145 cluster_stats_messages_received:278 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 主节点 - 5461-10922 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 主节点 - 10923-16383 (共 0 个槽位) abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441 主节点 - 0-5460 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.9.66:31952) 负责 0 个槽位: 5461-10922 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.8.48:31411) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.0.89:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl get secrets -l app.kubernetes.io/instance=rediscl-epxpxf` set secret: rediscl-epxpxf-shard-kss-account-default `kubectl get secrets rediscl-epxpxf-shard-kss-account-default -o jsonpath="***.data.username***"` `kubectl get secrets rediscl-epxpxf-shard-kss-account-default -o jsonpath="***.data.password***"` `kubectl get secrets rediscl-epxpxf-shard-kss-account-default -o jsonpath="***.data.port***"` DB_USERNAME:default;DB_PASSWORD:Xm507FZu07;DB_PORT:6379;DB_DATABASE: There are no pods gotten. describe cluster `kbcli cluster describe rediscl-epxpxf --namespace ns-hbjkr ` Name: rediscl-epxpxf Created Time: May 28,2025 11:38 UTC+0800 NAMESPACE CLUSTER-DEFINITION TOPOLOGY STATUS TERMINATION-POLICY ns-hbjkr Running WipeOut Endpoints: COMPONENT INTERNAL EXTERNAL shard(shard-4sx) rediscl-epxpxf-shard-4sx-redis-advertised-0.ns-hbjkr.svc.cluster.local:6379 ec2-35-93-160-91.us-west-2.compute.amazonaws.com:31411 rediscl-epxpxf-shard-4sx-redis-advertised-0.ns-hbjkr.svc.cluster.local:16379 ec2-35-93-160-91.us-west-2.compute.amazonaws.com:31995 rediscl-epxpxf-shard-4sx-redis-advertised-1.ns-hbjkr.svc.cluster.local:6379 ec2-35-93-160-91.us-west-2.compute.amazonaws.com:31841 rediscl-epxpxf-shard-4sx-redis-advertised-1.ns-hbjkr.svc.cluster.local:16379 ec2-35-93-160-91.us-west-2.compute.amazonaws.com:30786 shard(shard-58x) rediscl-epxpxf-shard-58x-redis-advertised-0.ns-hbjkr.svc.cluster.local:6379 ec2-35-93-160-91.us-west-2.compute.amazonaws.com:32441 rediscl-epxpxf-shard-58x-redis-advertised-0.ns-hbjkr.svc.cluster.local:16379 ec2-35-93-160-91.us-west-2.compute.amazonaws.com:32196 rediscl-epxpxf-shard-58x-redis-advertised-1.ns-hbjkr.svc.cluster.local:6379 ec2-35-93-160-91.us-west-2.compute.amazonaws.com:32651 rediscl-epxpxf-shard-58x-redis-advertised-1.ns-hbjkr.svc.cluster.local:16379 ec2-35-93-160-91.us-west-2.compute.amazonaws.com:30358 shard(shard-kss) rediscl-epxpxf-shard-kss-redis-advertised-0.ns-hbjkr.svc.cluster.local:6379 ec2-35-93-160-91.us-west-2.compute.amazonaws.com:31952 rediscl-epxpxf-shard-kss-redis-advertised-0.ns-hbjkr.svc.cluster.local:16379 ec2-35-93-160-91.us-west-2.compute.amazonaws.com:30805 rediscl-epxpxf-shard-kss-redis-advertised-1.ns-hbjkr.svc.cluster.local:6379 ec2-35-93-160-91.us-west-2.compute.amazonaws.com:30752 rediscl-epxpxf-shard-kss-redis-advertised-1.ns-hbjkr.svc.cluster.local:16379 ec2-35-93-160-91.us-west-2.compute.amazonaws.com:31230 Topology: COMPONENT SERVICE-VERSION INSTANCE ROLE STATUS AZ NODE CREATED-TIME shard(shard-4sx) 7.2.7 rediscl-epxpxf-shard-4sx-0 primary Running us-west-2a ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:38 UTC+0800 shard(shard-4sx) 7.2.7 rediscl-epxpxf-shard-4sx-1 secondary Running us-west-2a ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 11:39 UTC+0800 shard(shard-58x) 7.2.7 rediscl-epxpxf-shard-58x-0 primary Running us-west-2a ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:38 UTC+0800 shard(shard-58x) 7.2.7 rediscl-epxpxf-shard-58x-1 secondary Running us-west-2a ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:39 UTC+0800 shard(shard-kss) 7.2.7 rediscl-epxpxf-shard-kss-0 primary Running us-west-2a ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 11:38 UTC+0800 shard(shard-kss) 7.2.7 rediscl-epxpxf-shard-kss-1 secondary Running us-west-2a ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:39 UTC+0800 Resources Allocation: COMPONENT INSTANCE-TEMPLATE CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS shard 100m / 100m 512Mi / 512Mi data:1Gi kb-default-sc Images: COMPONENT COMPONENT-DEFINITION IMAGE shard redis-cluster-7-1.0.0-alpha.0 docker.io/apecloud/redis-stack-server:7.2.0-v14 docker.io/apecloud/agamotto:0.1.2-beta.1 Data Protection: BACKUP-REPO AUTO-BACKUP BACKUP-SCHEDULE BACKUP-METHOD BACKUP-RETENTION RECOVERABLE-TIME Show cluster events: kbcli cluster list-events -n ns-hbjkr rediscl-epxpxf get shard component name set shard component name:shard-58x `kbcli cluster label rediscl-epxpxf app.kubernetes.io/instance- --namespace ns-hbjkr ` label "app.kubernetes.io/instance" not found. `kbcli cluster label rediscl-epxpxf app.kubernetes.io/instance=rediscl-epxpxf --namespace ns-hbjkr ` `kbcli cluster label rediscl-epxpxf --list --namespace ns-hbjkr ` NAME NAMESPACE LABELS rediscl-epxpxf ns-hbjkr app.kubernetes.io/instance=rediscl-epxpxf label cluster app.kubernetes.io/instance=rediscl-epxpxf Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=rediscl-epxpxf --namespace ns-hbjkr ` `kbcli cluster label rediscl-epxpxf --list --namespace ns-hbjkr ` NAME NAMESPACE LABELS rediscl-epxpxf ns-hbjkr app.kubernetes.io/instance=rediscl-epxpxf case.name=kbcli.test1 label cluster case.name=kbcli.test1 Success `kbcli cluster label rediscl-epxpxf case.name=kbcli.test2 --overwrite --namespace ns-hbjkr ` `kbcli cluster label rediscl-epxpxf --list --namespace ns-hbjkr ` NAME NAMESPACE LABELS rediscl-epxpxf ns-hbjkr app.kubernetes.io/instance=rediscl-epxpxf case.name=kbcli.test2 label cluster case.name=kbcli.test2 Success `kbcli cluster label rediscl-epxpxf case.name- --namespace ns-hbjkr ` `kbcli cluster label rediscl-epxpxf --list --namespace ns-hbjkr ` NAME NAMESPACE LABELS rediscl-epxpxf ns-hbjkr app.kubernetes.io/instance=rediscl-epxpxf delete cluster label case.name Success cluster connect kbcli unsupported engine type: shard-58x test failover networkduplicate check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-rediscl-epxpxf" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-rediscl-epxpxf" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkduplicate-rediscl-epxpxf namespace: ns-hbjkr spec: selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-0 mode: all action: duplicate duplicate: duplicate: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkduplicate-rediscl-epxpxf.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkduplicate-rediscl-epxpxf created apply test-chaos-mesh-networkduplicate-rediscl-epxpxf.yaml Success `rm -rf test-chaos-mesh-networkduplicate-rediscl-epxpxf.yaml` networkduplicate chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748403905504 2 connected 5461-10922 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748403906000 2 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748403904000 3 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748403905504 1 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748403905508 1 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748403906575 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:529 cluster_stats_messages_pong_sent:528 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1058 cluster_stats_messages_ping_received:527 cluster_stats_messages_pong_received:530 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:1058 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748403911000 1 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748403911521 1 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748403911000 2 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748403911000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748403911822 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748403910000 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:527 cluster_stats_messages_pong_sent:519 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1047 cluster_stats_messages_ping_received:519 cluster_stats_messages_pong_received:528 cluster_stats_messages_received:1047 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748403914000 1 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748403915000 2 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748403914554 2 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748403915256 1 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748403914554 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748403916263 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:551 cluster_stats_messages_pong_sent:579 cluster_stats_messages_sent:1130 cluster_stats_messages_ping_received:576 cluster_stats_messages_pong_received:551 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:1130 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748403921495 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748403920000 1 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748403919484 2 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748403920488 3 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748403921000 1 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748403920000 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:560 cluster_stats_messages_pong_sent:548 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1109 cluster_stats_messages_ping_received:548 cluster_stats_messages_pong_received:561 cluster_stats_messages_received:1109 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748403924000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748403925505 1 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748403924802 1 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748403925806 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748403925000 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748403924601 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:566 cluster_stats_messages_pong_sent:570 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1137 cluster_stats_messages_ping_received:569 cluster_stats_messages_pong_received:567 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:1137 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748403930000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748403930237 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748403929735 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748403929000 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748403930739 1 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748403931241 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:578 cluster_stats_messages_pong_sent:575 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1154 cluster_stats_messages_ping_received:575 cluster_stats_messages_pong_received:579 cluster_stats_messages_received:1154 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 主节点 - 5461-10922 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 主节点 - 10923-16383 (共 0 个槽位) abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441 主节点 - 0-5460 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.9.66:31952) 负责 0 个槽位: 5461-10922 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.8.48:31411) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.0.89:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-rediscl-epxpxf" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-rediscl-epxpxf" not found check failover pod name failover pod name:rediscl-epxpxf-shard-58x-0 failover networkduplicate Success test failover dnserror check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-rediscl-epxpxf" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-rediscl-epxpxf" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnserror-rediscl-epxpxf namespace: ns-hbjkr spec: selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-0 mode: all action: error duration: 2m `kubectl apply -f test-chaos-mesh-dnserror-rediscl-epxpxf.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnserror-rediscl-epxpxf created apply test-chaos-mesh-dnserror-rediscl-epxpxf.yaml Success `rm -rf test-chaos-mesh-dnserror-rediscl-epxpxf.yaml` dnserror chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404138417 2 connected 5461-10922 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404137113 2 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748404136000 3 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404137000 1 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748404138000 1 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404137416 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:945 cluster_stats_messages_pong_sent:951 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1897 cluster_stats_messages_ping_received:950 cluster_stats_messages_pong_received:946 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:1897 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404143534 1 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748404143000 1 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404142000 2 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404142000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404142025 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404143030 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:3 cluster_stats_messages_ping_sent:956 cluster_stats_messages_pong_sent:936 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1893 cluster_stats_messages_ping_received:936 cluster_stats_messages_pong_received:957 cluster_stats_messages_received:1893 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748404147000 1 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404147000 2 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404147117 2 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748404148000 1 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404148122 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404148021 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:980 cluster_stats_messages_pong_sent:1000 cluster_stats_messages_sent:1980 cluster_stats_messages_ping_received:997 cluster_stats_messages_pong_received:980 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:1980 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404152024 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404151000 1 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404152000 2 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404151000 3 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748404150000 1 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404152325 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:980 cluster_stats_messages_pong_sent:961 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1942 cluster_stats_messages_ping_received:961 cluster_stats_messages_pong_received:981 cluster_stats_messages_received:1942 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748404156000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404156088 1 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748404157596 1 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404156088 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404157596 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404157000 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:995 cluster_stats_messages_pong_sent:1002 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1998 cluster_stats_messages_ping_received:1001 cluster_stats_messages_pong_received:995 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:1997 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404161512 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404161515 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404161000 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404160000 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404162000 1 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748404162818 1 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:2 cluster_stats_messages_ping_sent:996 cluster_stats_messages_pong_sent:1006 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:2003 cluster_stats_messages_ping_received:1006 cluster_stats_messages_pong_received:997 cluster_stats_messages_received:2003 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 主节点 - 5461-10922 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 主节点 - 10923-16383 (共 0 个槽位) abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441 主节点 - 0-5460 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.9.66:31952) 负责 0 个槽位: 5461-10922 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.8.48:31411) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.0.89:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节�� ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-rediscl-epxpxf" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-rediscl-epxpxf" not found check failover pod name failover pod name:rediscl-epxpxf-shard-58x-0 failover dnserror Success test failover networklossover check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-rediscl-epxpxf" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-rediscl-epxpxf" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-rediscl-epxpxf namespace: ns-hbjkr spec: selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-0 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-rediscl-epxpxf.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-rediscl-epxpxf created apply test-chaos-mesh-networklossover-rediscl-epxpxf.yaml Success `rm -rf test-chaos-mesh-networklossover-rediscl-epxpxf.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404372552 2 connected 5461-10922 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404372445 2 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748404371000 3 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748404371000 4 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404371441 4 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404371000 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:3 cluster_stats_messages_ping_sent:1324 cluster_stats_messages_pong_sent:1344 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:2675 cluster_stats_messages_ping_received:1343 cluster_stats_messages_pong_received:1325 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:2671 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748404377247 4 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404375742 4 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404376000 2 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404376000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404377549 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404376239 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:3 cluster_stats_messages_ping_sent:1361 cluster_stats_messages_pong_sent:1320 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:2682 cluster_stats_messages_ping_received:1320 cluster_stats_messages_pong_received:1362 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:2685 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748404380000 4 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404381000 2 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404381000 2 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404380058 4 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404381566 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404381566 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:4 cluster_stats_messages_ping_sent:1191 cluster_stats_messages_pong_sent:1212 cluster_stats_messages_sent:2403 cluster_stats_messages_ping_received:1209 cluster_stats_messages_pong_received:1186 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:2398 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404387208 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748404386706 4 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404386203 2 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404385594 3 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748404385000 4 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404386605 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:4 cluster_stats_messages_ping_sent:1396 cluster_stats_messages_pong_sent:1366 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:2768 cluster_stats_messages_ping_received:1361 cluster_stats_messages_pong_received:1396 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:2761 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748404389000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748404391631 4 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404391000 4 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404391000 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404391531 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404391534 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:2 cluster_stats_messages_ping_sent:1395 cluster_stats_messages_pong_sent:1410 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:2812 cluster_stats_messages_ping_received:1409 cluster_stats_messages_pong_received:1396 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:2808 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404394326 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404396338 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404396539 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404395000 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748404395533 4 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404395398 4 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:2 cluster_stats_messages_ping_sent:1391 cluster_stats_messages_pong_sent:1414 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:2806 cluster_stats_messages_ping_received:1414 cluster_stats_messages_pong_received:1392 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:2809 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 主节点 - 5461-10922 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 主节点 - 10923-16383 (共 0 个槽位) abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441 从节点 937c7ef861bd0c1128f152b847059444b1dd5291 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 主节点 - 0-5460 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.9.66:31952) 负责 0 个槽位: 5461-10922 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.8.48:31411) 负责 0 个槽位: 10923-16383 节点 937c7ef861bd0c1128f152b847059444b1dd5291 (172.31.8.48:32651) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-rediscl-epxpxf" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-rediscl-epxpxf" not found check failover pod name failover pod name:rediscl-epxpxf-shard-58x-1 failover networklossover Success test failover fullcpu check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-rediscl-epxpxf" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-rediscl-epxpxf" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-fullcpu-rediscl-epxpxf namespace: ns-hbjkr spec: selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-1 mode: all stressors: cpu: workers: 100 load: 100 duration: 2m `kubectl apply -f test-chaos-mesh-fullcpu-rediscl-epxpxf.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-fullcpu-rediscl-epxpxf created apply test-chaos-mesh-fullcpu-rediscl-epxpxf.yaml Success `rm -rf test-chaos-mesh-fullcpu-rediscl-epxpxf.yaml` fullcpu chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404604000 2 connected 5461-10922 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404605000 2 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748404604000 3 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748404605000 4 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 1748404606466 1748404604000 4 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404605464 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:3 cluster_stats_messages_ping_sent:1737 cluster_stats_messages_pong_sent:1768 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:3512 cluster_stats_messages_ping_received:1767 cluster_stats_messages_pong_received:1734 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:3504 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748404609014 4 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404610000 4 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404610521 2 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404608000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404610000 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404609516 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:3 cluster_stats_messages_ping_sent:1784 cluster_stats_messages_pong_sent:1724 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:3509 cluster_stats_messages_ping_received:1724 cluster_stats_messages_pong_received:1782 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:3509 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748404615000 4 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404614000 2 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404615532 2 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404614000 4 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404614527 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404614024 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:4 cluster_stats_messages_ping_sent:1605 cluster_stats_messages_pong_sent:1633 cluster_stats_messages_sent:3238 cluster_stats_messages_ping_received:1630 cluster_stats_messages_pong_received:1599 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:3232 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404619101 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748404620106 4 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404619000 2 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404619603 3 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748404619000 4 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404619000 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:4 cluster_stats_messages_ping_sent:1730 cluster_stats_messages_pong_sent:1725 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:3461 cluster_stats_messages_ping_received:1720 cluster_stats_messages_pong_received:1730 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:3454 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748404624000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748404624398 4 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404623392 4 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404624699 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404625402 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404624202 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:2 cluster_stats_messages_ping_sent:1828 cluster_stats_messages_pong_sent:1813 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:3648 cluster_stats_messages_ping_received:1812 cluster_stats_messages_pong_received:1827 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:3642 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404629630 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404629127 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404629000 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404628000 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748404630133 4 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404629000 4 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:4 cluster_my_epoch:2 cluster_stats_messages_ping_sent:1818 cluster_stats_messages_pong_sent:1840 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:3659 cluster_stats_messages_ping_received:1840 cluster_stats_messages_pong_received:1816 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:3659 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 主节点 - 5461-10922 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 主节点 - 10923-16383 (共 0 个槽位) abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441 从节点 937c7ef861bd0c1128f152b847059444b1dd5291 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 主节点 - 0-5460 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.9.66:31952) 负责 0 个槽位: 5461-10922 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.8.48:31411) 负责 0 个槽位: 10923-16383 节点 937c7ef861bd0c1128f152b847059444b1dd5291 (172.31.8.48:32651) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-rediscl-epxpxf" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-rediscl-epxpxf" not found check failover pod name failover pod name:rediscl-epxpxf-shard-58x-1 failover fullcpu Success test failover delete pod:rediscl-epxpxf-shard-58x-1 `kubectl delete pod rediscl-epxpxf-shard-58x-1 --namespace ns-hbjkr ` pod "rediscl-epxpxf-shard-58x-1" deleted check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Updating May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:58 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404742055 2 connected 5461-10922 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404740545 2 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748404740000 3 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404741000 5 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748404740548 5 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404741552 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:5 cluster_my_epoch:3 cluster_stats_messages_ping_sent:1977 cluster_stats_messages_pong_sent:2018 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:4003 cluster_stats_messages_ping_received:2017 cluster_stats_messages_pong_received:1974 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:3996 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404746200 5 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748404745000 5 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404745194 2 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404745000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404747109 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 1748404747207 1748404745000 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:5 cluster_my_epoch:3 cluster_stats_messages_ping_sent:2038 cluster_stats_messages_pong_sent:1963 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:4007 cluster_stats_messages_ping_received:1963 cluster_stats_messages_pong_received:2036 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:4003 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748404750000 5 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404750614 2 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404751619 2 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748404750000 5 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404751017 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404750000 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:5 cluster_my_epoch:5 cluster_stats_messages_ping_sent:1849 cluster_stats_messages_pong_sent:1895 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:3750 cluster_stats_messages_ping_received:1886 cluster_stats_messages_pong_received:1842 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:3735 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404755573 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404756578 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404755000 5 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404756000 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404755000 2 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748404755000 5 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:5 cluster_my_epoch:5 cluster_stats_messages_ping_sent:84 cluster_stats_messages_pong_sent:58 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:143 cluster_stats_messages_ping_received:58 cluster_stats_messages_pong_received:85 cluster_stats_messages_received:143 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748404759000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404761308 5 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748404760303 5 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404760805 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404761308 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404760000 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:5 cluster_my_epoch:2 cluster_stats_messages_ping_sent:2069 cluster_stats_messages_pong_sent:2055 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:4132 cluster_stats_messages_ping_received:2054 cluster_stats_messages_pong_received:2068 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:4127 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748404765108 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748404764103 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748404765000 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748404765000 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748404765511 5 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748404766114 5 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:5 cluster_my_epoch:2 cluster_stats_messages_ping_sent:2053 cluster_stats_messages_pong_sent:2093 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:4147 cluster_stats_messages_ping_received:2093 cluster_stats_messages_pong_received:2051 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:4149 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 主节点 - 5461-10922 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 主节点 - 10923-16383 (共 0 个槽位) abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441 主节点 - 0-5460 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.9.66:31952) 负责 0 个槽位: 5461-10922 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.8.48:31411) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.0.89:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success check failover pod name failover pod name:rediscl-epxpxf-shard-58x-0 failover Success `kubectl get secrets -l app.kubernetes.io/instance=rediscl-epxpxf` set secret: rediscl-epxpxf-shard-58x-account-default `kubectl get secrets rediscl-epxpxf-shard-58x-account-default -o jsonpath="***.data.username***"` `kubectl get secrets rediscl-epxpxf-shard-58x-account-default -o jsonpath="***.data.password***"` `kubectl get secrets rediscl-epxpxf-shard-58x-account-default -o jsonpath="***.data.port***"` DB_USERNAME:default;DB_PASSWORD:Xm507FZu07;DB_PORT:6379;DB_DATABASE: apiVersion: v1 kind: Pod metadata: name: benchtest-rediscl-epxpxf namespace: ns-hbjkr spec: containers: - name: test-benchmark imagePullPolicy: IfNotPresent image: docker.io/apecloud/redis-benchmark:latest args: - "-h" - "rediscl-epxpxf-shard-58x-redis-advertised-0.ns-hbjkr.svc.cluster.local" - "-p" - "6379" - "-a" - "Xm507FZu07" - "-n" - "1000" - "-c" - "2" - "--cluster" - "-q" restartPolicy: Never `kubectl apply -f benchtest-rediscl-epxpxf.yaml` pod/benchtest-rediscl-epxpxf created apply benchtest-rediscl-epxpxf.yaml Success `rm -rf benchtest-rediscl-epxpxf.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE benchtest-rediscl-epxpxf 0/1 ContainerCreating 0 1s pod_status:NAME READY STATUS RESTARTS AGE benchtest-rediscl-epxpxf 1/1 Running 0 6s pod_status:NAME READY STATUS RESTARTS AGE benchtest-rediscl-epxpxf 1/1 Running 0 11s pod_status:NAME READY STATUS RESTARTS AGE benchtest-rediscl-epxpxf 1/1 Running 0 17s check pod benchtest-rediscl-epxpxf status done pod_status:NAME READY STATUS RESTARTS AGE benchtest-rediscl-epxpxf 0/1 Completed 0 23s `kubectl logs benchtest-rediscl-epxpxf --tail 30 --namespace ns-hbjkr ` Cluster has 3 master nodes: Master 0: abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441 Master 1: 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 Master 2: 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 PING_INLINE: rps=0.0 (overall: 1000.0) avg_msec=0.291 (overall: 0.174) PING_INLINE: rps=3713.1 (overall: 3702.4) avg_msec=0.452 (overall: 0.451) PING_INLINE: 1992.03 requests per second, p50=0.239 msec PING_MBULK: rps=15.9 (overall: 4000.0) avg_msec=0.290 (overall: 0.362) PING_MBULK: rps=3370.5 (overall: 3373.0) avg_msec=0.497 (overall: 0.496) PING_MBULK: 1988.07 requests per second, p50=0.247 msec SET: rps=7.9 (overall: 2000.0) avg_msec=0.433 (overall: 0.360) SET: rps=1865.1 (overall: 1865.6) avg_msec=0.808 (overall: 0.806) SET: rps=1452.0 (overall: 1660.0) avg_msec=1.592 (overall: 1.148) SET: 1328.02 requests per second, p50=0.271 msec GET: rps=11.9 (overall: 3000.0) avg_msec=0.259 (overall: 0.261) GET: rps=3585.7 (overall: 3583.3) avg_msec=0.538 (overall: 0.537) GET: 1992.03 requests per second, p50=0.247 msec INCR: rps=4.0 (overall: 1000.0) avg_msec=1.068 (overall: 1.068) INCR: rps=2585.7 (overall: 2579.4) avg_msec=0.721 (overall: 0.722) INCR: 1992.03 requests per second, p50=0.359 msec LPUSH: rps=7.9 (overall: 2000.0) avg_msec=0.363 (overall: 0.544) LPUSH: rps=2529.9 (overall: 2527.8) avg_msec=0.735 (overall: 0.733) LPUSH: rps=1350.6 (overall: 1940.4) avg_msec=1.371 (overall: 0.955) LPUSH: 1329.79 requests per second, p50=0.287 msec RPUSH: rps=4.0 (overall: inf) avg_msec=0.376 (overall: 0.266) RPUSH: rps=2559.5 (overall: 2563.5) avg_msec=0.707 (overall: 0.707) RPUSH: 1992.03 requests per second, p50=0.287 msec LPOP: rps=7.9 (overall: 2000.0) avg_msec=0.445 (overall: 0.668) LPOP: rps=2478.1 (overall: 2476.2) avg_msec=0.719 (overall: 0.718) LPOP: 1992.03 requests per second, p50=0.327 msec RPOP: rps=7.9 (overall: 2000.0) avg_msec=0.339 (overall: 0.508) RPOP: rps=2533.9 (overall: 2531.7) avg_msec=0.728 (overall: 0.727) RPOP: 1988.07 requests per second, p50=0.271 msec SADD: rps=7.9 (overall: 2000.0) avg_msec=0.327 (overall: 0.352) SADD: rps=3972.1 (overall: 3964.3) avg_msec=0.445 (overall: 0.445) SADD: 1992.03 requests per second, p50=0.231 msec HSET: rps=7.9 (overall: 2000.0) avg_msec=0.312 (overall: 0.468) HSET: rps=2613.5 (overall: 2611.1) avg_msec=0.673 (overall: 0.673) HSET: 1992.03 requests per second, p50=0.279 msec SPOP: rps=8.0 (overall: inf) avg_msec=0.492 (overall: 0.492) SPOP: rps=3972.1 (overall: 3980.1) avg_msec=0.437 (overall: 0.438) SPOP: 1992.03 requests per second, p50=0.223 msec ZADD: rps=7.9 (overall: 2000.0) avg_msec=0.303 (overall: 0.304) ZADD: rps=3865.1 (overall: 3857.7) avg_msec=0.450 (overall: 0.450) ZADD: 1992.03 requests per second, p50=0.215 msec ZPOPMIN: rps=8.0 (overall: 2000.0) avg_msec=0.725 (overall: 1.088) ZPOPMIN: rps=3956.3 (overall: 3948.6) avg_msec=0.425 (overall: 0.426) ZPOPMIN: 1996.01 requests per second, p50=0.223 msec LPUSH (needed to benchmark LRANGE): rps=8.0 (overall: 2000.0) avg_msec=0.415 (overall: 0.333) LPUSH (needed to benchmark LRANGE): rps=2900.4 (overall: 2896.8) avg_msec=0.590 (overall: 0.588) LPUSH (needed to benchmark LRANGE): 1992.03 requests per second, p50=0.255 msec LRANGE_100 (first 100 elements): rps=7.9 (overall: 2000.0) avg_msec=0.432 (overall: 0.432) LRANGE_100 (first 100 elements): rps=3008.0 (overall: 3004.0) avg_msec=0.520 (overall: 0.519) LRANGE_100 (first 100 elements): 1992.03 requests per second, p50=0.223 msec LRANGE_300 (first 300 elements): rps=4.0 (overall: 1000.0) avg_msec=0.596 (overall: 0.596) LRANGE_300 (first 300 elements): rps=2685.3 (overall: 2678.6) avg_msec=0.684 (overall: 0.685) LRANGE_300 (first 300 elements): 1988.07 requests per second, p50=0.279 msec LRANGE_500 (first 500 elements): rps=7.9 (overall: 2000.0) avg_msec=0.285 (overall: 0.428) LRANGE_500 (first 500 elements): rps=2203.2 (overall: 2202.4) avg_msec=0.807 (overall: 0.804) LRANGE_500 (first 500 elements): rps=1768.9 (overall: 1986.1) avg_msec=0.895 (overall: 0.845) LRANGE_500 (first 500 elements): 1328.02 requests per second, p50=0.287 msec LRANGE_600 (first 600 elements): rps=7.9 (overall: 2000.0) avg_msec=0.288 (overall: 0.288) LRANGE_600 (first 600 elements): rps=1968.1 (overall: 1968.3) avg_msec=0.763 (overall: 0.761) LRANGE_600 (first 600 elements): rps=1804.8 (overall: 1886.7) avg_msec=1.166 (overall: 0.954) LRANGE_600 (first 600 elements): 1328.02 requests per second, p50=0.295 msec MSET (10 keys): rps=0.0 (overall: 0.0) avg_msec=-nan (overall: -nan) MSET (10 keys): rps=2179.3 (overall: 2170.6) avg_msec=0.871 (overall: 0.871) MSET (10 keys): rps=1346.6 (overall: 1759.4) avg_msec=1.211 (overall: 1.001) MSET (10 keys): 1328.02 requests per second, p50=0.295 msec XADD: rps=7.9 (overall: 2000.0) avg_msec=0.325 (overall: 0.347) XADD: rps=1581.7 (overall: 1583.3) avg_msec=0.952 (overall: 0.947) XADD: rps=1960.2 (overall: 1771.4) avg_msec=1.247 (overall: 1.112) XADD: 1329.79 requests per second, p50=0.319 msec `kubectl delete pod benchtest-rediscl-epxpxf --force --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "benchtest-rediscl-epxpxf" force deleted test failover networkbandwidthover check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-rediscl-epxpxf" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-rediscl-epxpxf" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkbandwidthover-rediscl-epxpxf namespace: ns-hbjkr spec: selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-0 action: bandwidth mode: all bandwidth: rate: '1bps' limit: 20971520 buffer: 10000 duration: 2m `kubectl apply -f test-chaos-mesh-networkbandwidthover-rediscl-epxpxf.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkbandwidthover-rediscl-epxpxf created apply test-chaos-mesh-networkbandwidthover-rediscl-epxpxf.yaml Success `rm -rf test-chaos-mesh-networkbandwidthover-rediscl-epxpxf.yaml` networkbandwidthover chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:58 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405065576 2 connected 5461-10922 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405064571 2 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405065000 3 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748405065078 6 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405064071 6 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405066082 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:3 cluster_stats_messages_ping_sent:2548 cluster_stats_messages_pong_sent:2576 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-ack_sent:3 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:5139 cluster_stats_messages_ping_received:2575 cluster_stats_messages_pong_received:2545 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:3 cluster_stats_messages_received:5127 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748405070413 6 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405069000 6 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405070000 2 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405068000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405070000 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405070918 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:3 cluster_stats_messages_ping_sent:2601 cluster_stats_messages_pong_sent:2516 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:5123 cluster_stats_messages_ping_received:2516 cluster_stats_messages_pong_received:2598 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:3 cluster_stats_messages_received:5121 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748405074000 6 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405076000 2 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405074153 2 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405076165 6 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405076064 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405074655 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:6 cluster_stats_messages_ping_sent:2227 cluster_stats_messages_pong_sent:2271 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:4504 cluster_stats_messages_ping_received:2262 cluster_stats_messages_pong_received:2215 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_update_received:1 cluster_stats_messages_received:4485 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405080126 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405079000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748405079000 6 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405080000 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405081131 2 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405079000 6 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:6 cluster_stats_messages_ping_sent:654 cluster_stats_messages_pong_sent:622 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1282 cluster_stats_messages_ping_received:617 cluster_stats_messages_pong_received:654 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:1275 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405083000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748405084000 6 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405085000 6 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405085070 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405085073 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405085000 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:2 cluster_stats_messages_ping_sent:2615 cluster_stats_messages_pong_sent:2611 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-ack_sent:3 cluster_stats_messages_sent:5240 cluster_stats_messages_ping_received:2610 cluster_stats_messages_pong_received:2614 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:3 cluster_stats_messages_received:5231 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405089506 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405089000 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405090000 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405089000 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748405089404 6 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405090411 6 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:2 cluster_stats_messages_ping_sent:2600 cluster_stats_messages_pong_sent:2654 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:5255 cluster_stats_messages_ping_received:2654 cluster_stats_messages_pong_received:2598 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:3 cluster_stats_messages_received:5260 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 主节点 - 5461-10922 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 主节点 - 10923-16383 (共 0 个槽位) abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441 从节点 937c7ef861bd0c1128f152b847059444b1dd5291 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 主节点 - 0-5460 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.9.66:31952) 负责 0 个槽位: 5461-10922 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.8.48:31411) 负责 0 个槽位: 10923-16383 节点 937c7ef861bd0c1128f152b847059444b1dd5291 (172.31.8.48:32651) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-rediscl-epxpxf" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-rediscl-epxpxf" not found check failover pod name failover pod name:rediscl-epxpxf-shard-58x-1 failover networkbandwidthover Success test failover networkcorruptover check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-rediscl-epxpxf" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-rediscl-epxpxf" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorruptover-rediscl-epxpxf namespace: ns-hbjkr spec: selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-1 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorruptover-rediscl-epxpxf.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-rediscl-epxpxf created apply test-chaos-mesh-networkcorruptover-rediscl-epxpxf.yaml Success `rm -rf test-chaos-mesh-networkcorruptover-rediscl-epxpxf.yaml` networkcorruptover chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:58 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405299781 2 connected 5461-10922 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405299000 2 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405299000 3 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405299000 7 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405299275 7 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405299181 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:3 cluster_stats_messages_ping_sent:2950 cluster_stats_messages_pong_sent:2977 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-ack_sent:4 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:5948 cluster_stats_messages_ping_received:2976 cluster_stats_messages_pong_received:2940 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_received:5925 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405303321 7 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405303000 7 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405304325 2 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405302000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405303000 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405303522 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:3 cluster_stats_messages_ping_sent:3003 cluster_stats_messages_pong_sent:2904 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:5913 cluster_stats_messages_ping_received:2904 cluster_stats_messages_pong_received:2994 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_received:5908 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405309000 7 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405308045 2 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405308000 2 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405309051 7 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405309600 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405309600 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:2637 cluster_stats_messages_pong_sent:2691 cluster_stats_messages_auth-req_sent:10 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:5339 cluster_stats_messages_ping_received:2677 cluster_stats_messages_pong_received:2616 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-ack_received:4 cluster_stats_messages_update_received:1 cluster_stats_messages_received:5305 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405314000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405313084 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405313084 7 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405314693 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405313000 2 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405313000 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:890 cluster_stats_messages_pong_sent:873 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1769 cluster_stats_messages_ping_received:868 cluster_stats_messages_pong_received:867 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:1741 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405318000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405319000 7 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405318749 7 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405319552 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405317543 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405319000 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:2 cluster_stats_messages_ping_sent:3015 cluster_stats_messages_pong_sent:3017 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-ack_sent:4 cluster_stats_messages_sent:6052 cluster_stats_messages_ping_received:3016 cluster_stats_messages_pong_received:3005 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_received:6030 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405322004 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405323000 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405324017 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405322000 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405323008 7 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405322509 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:2 cluster_stats_messages_ping_sent:3006 cluster_stats_messages_pong_sent:3040 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:6047 cluster_stats_messages_ping_received:3040 cluster_stats_messages_pong_received:2998 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_received:6049 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 主节点 - 5461-10922 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 主节点 - 10923-16383 (共 0 个槽位) abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441 主节点 - 0-5460 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.9.66:31952) 负责 0 个槽位: 5461-10922 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.8.48:31411) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.0.89:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-rediscl-epxpxf" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-rediscl-epxpxf" not found check failover pod name failover pod name:rediscl-epxpxf-shard-58x-0 failover networkcorruptover Success test failover networkdelay check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-rediscl-epxpxf" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-rediscl-epxpxf" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkdelay-rediscl-epxpxf namespace: ns-hbjkr spec: selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-0 mode: all action: delay delay: latency: 2000ms correlation: '100' jitter: 0ms direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkdelay-rediscl-epxpxf.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkdelay-rediscl-epxpxf created apply test-chaos-mesh-networkdelay-rediscl-epxpxf.yaml Success `rm -rf test-chaos-mesh-networkdelay-rediscl-epxpxf.yaml` networkdelay chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:58 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405532000 2 connected 5461-10922 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405533201 2 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405532000 3 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405532699 7 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405532000 7 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405532197 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:3 cluster_stats_messages_ping_sent:3366 cluster_stats_messages_pong_sent:3405 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-ack_sent:4 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:6792 cluster_stats_messages_ping_received:3404 cluster_stats_messages_pong_received:3355 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_received:6768 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405537000 7 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405537000 7 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405537000 2 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405536000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405537711 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405537000 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:3 cluster_stats_messages_ping_sent:3420 cluster_stats_messages_pong_sent:3323 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:6749 cluster_stats_messages_ping_received:3323 cluster_stats_messages_pong_received:3411 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_received:6744 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405541000 7 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405542000 2 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405542703 2 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405543204 7 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405542600 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405541646 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:2992 cluster_stats_messages_pong_sent:3031 cluster_stats_messages_auth-req_sent:10 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:6034 cluster_stats_messages_ping_received:3017 cluster_stats_messages_pong_received:2971 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-ack_received:4 cluster_stats_messages_update_received:1 cluster_stats_messages_received:6000 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405547387 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405546000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405546000 7 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405546382 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405547589 2 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405546000 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:1315 cluster_stats_messages_pong_sent:1294 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:2615 cluster_stats_messages_ping_received:1289 cluster_stats_messages_pong_received:1291 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:2586 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405551000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405552000 7 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405551537 7 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405552542 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405551034 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405552542 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:2 cluster_stats_messages_ping_sent:3432 cluster_stats_messages_pong_sent:3426 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-ack_sent:4 cluster_stats_messages_sent:6878 cluster_stats_messages_ping_received:3425 cluster_stats_messages_pong_received:3421 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_received:6855 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405557523 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405557000 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405556717 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405556000 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405557522 7 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405556000 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:2 cluster_stats_messages_ping_sent:3414 cluster_stats_messages_pong_sent:3460 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:6875 cluster_stats_messages_ping_received:3460 cluster_stats_messages_pong_received:3406 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_received:6877 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 主节点 - 5461-10922 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 主节点 - 10923-16383 (共 0 个槽位) abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441 主节点 - 0-5460 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.9.66:31952) 负责 0 个槽位: 5461-10922 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.8.48:31411) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.0.89:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-rediscl-epxpxf" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-rediscl-epxpxf" not found check failover pod name failover pod name:rediscl-epxpxf-shard-58x-0 failover networkdelay Success test failover networkpartition check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-rediscl-epxpxf" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-rediscl-epxpxf" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkpartition-rediscl-epxpxf namespace: ns-hbjkr spec: selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-0 action: partition mode: all target: mode: all selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-1 direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkpartition-rediscl-epxpxf.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkpartition-rediscl-epxpxf created apply test-chaos-mesh-networkpartition-rediscl-epxpxf.yaml Success `rm -rf test-chaos-mesh-networkpartition-rediscl-epxpxf.yaml` networkpartition chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:58 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405767000 2 connected 5461-10922 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405768573 2 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405767000 3 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405768072 7 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405767017 7 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405768574 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:3 cluster_stats_messages_ping_sent:3799 cluster_stats_messages_pong_sent:3834 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-ack_sent:4 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:7654 cluster_stats_messages_ping_received:3833 cluster_stats_messages_pong_received:3788 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_received:7630 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405771591 7 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405771000 7 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405771990 2 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405771000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405771000 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405772998 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:3 cluster_stats_messages_ping_sent:3853 cluster_stats_messages_pong_sent:3749 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:7608 cluster_stats_messages_ping_received:3749 cluster_stats_messages_pong_received:3844 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_received:7603 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405774000 7 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405776000 2 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405776439 2 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405777445 7 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405776000 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405776000 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:3413 cluster_stats_messages_pong_sent:3468 cluster_stats_messages_auth-req_sent:10 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:6892 cluster_stats_messages_ping_received:3454 cluster_stats_messages_pong_received:3392 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-ack_received:4 cluster_stats_messages_update_received:1 cluster_stats_messages_received:6858 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405781002 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405781000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405782009 7 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405780600 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405781000 2 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405781000 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:7 cluster_stats_messages_ping_sent:1759 cluster_stats_messages_pong_sent:1719 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:3484 cluster_stats_messages_ping_received:1714 cluster_stats_messages_pong_received:1735 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:3455 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405787000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405787521 7 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405786415 7 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405786112 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405787000 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405787420 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:2 cluster_stats_messages_ping_sent:3850 cluster_stats_messages_pong_sent:3846 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-ack_sent:4 cluster_stats_messages_sent:7716 cluster_stats_messages_ping_received:3845 cluster_stats_messages_pong_received:3839 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_received:7693 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405790308 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405792018 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405791315 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405791000 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405791000 7 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748405792320 7 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:7 cluster_my_epoch:2 cluster_stats_messages_ping_sent:3831 cluster_stats_messages_pong_sent:3887 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:7719 cluster_stats_messages_ping_received:3887 cluster_stats_messages_pong_received:3823 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_received:7721 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 主节点 - 5461-10922 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 主节点 - 10923-16383 (共 0 个槽位) abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441 主节点 - 0-5460 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.9.66:31952) 负责 0 个槽位: 5461-10922 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.8.48:31411) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.0.89:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-rediscl-epxpxf" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-rediscl-epxpxf" not found check failover pod name failover pod name:rediscl-epxpxf-shard-58x-0 failover networkpartition Success test failover kill1 check node drain check node drain success `kill 1` Defaulted container "redis-cluster" out of: redis-cluster, metrics, kbagent, init-dbctl (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file exec return message: check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Updating May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf cluster_status:Updating cluster_status:Updating cluster_status:Failed cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 11:39 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 11:58 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 11:38 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405912000 2 connected 5461-10922 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405912195 2 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405911000 3 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748405910184 8 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405910585 8 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405911189 3 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:8 cluster_my_epoch:3 cluster_stats_messages_ping_sent:4047 cluster_stats_messages_pong_sent:4084 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-ack_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:8153 cluster_stats_messages_ping_received:4083 cluster_stats_messages_pong_received:4036 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:5 cluster_stats_messages_received:8130 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748405916000 8 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405916824 8 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405916000 2 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405915000 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405916522 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405916525 3 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:8 cluster_my_epoch:3 cluster_stats_messages_ping_sent:4110 cluster_stats_messages_pong_sent:4003 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:8120 cluster_stats_messages_ping_received:4003 cluster_stats_messages_pong_received:4101 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:5 cluster_stats_messages_received:8116 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405921162 3 connected 10923-16383 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405919554 3 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748405920000 8 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405920000 8 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405920659 2 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405920000 2 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:8 cluster_my_epoch:8 cluster_stats_messages_ping_sent:61 cluster_stats_messages_pong_sent:43 cluster_stats_messages_sent:104 cluster_stats_messages_ping_received:43 cluster_stats_messages_pong_received:61 cluster_stats_messages_received:104 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405926172 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405925111 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748405925668 8 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405926172 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405925112 2 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405925000 8 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:8 cluster_my_epoch:8 cluster_stats_messages_ping_sent:2014 cluster_stats_messages_pong_sent:1987 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-req_sent:10 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:4018 cluster_stats_messages_ping_received:1977 cluster_stats_messages_pong_received:1989 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:4 cluster_stats_messages_received:3974 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748405929000 2 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748405930506 8 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405929601 8 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405931511 3 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405930000 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405930707 2 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:8 cluster_my_epoch:2 cluster_stats_messages_ping_sent:4100 cluster_stats_messages_pong_sent:4112 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-ack_sent:5 cluster_stats_messages_sent:8233 cluster_stats_messages_ping_received:4111 cluster_stats_messages_pong_received:4089 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:5 cluster_stats_messages_received:8211 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748405935401 3 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748405934000 2 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748405934000 3 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748405935000 2 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748405934377 8 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748405934578 8 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:8 cluster_my_epoch:2 cluster_stats_messages_ping_sent:4082 cluster_stats_messages_pong_sent:4140 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:8224 cluster_stats_messages_ping_received:4140 cluster_stats_messages_pong_received:4074 cluster_stats_messages_fail_received:8 cluster_stats_messages_auth-req_received:5 cluster_stats_messages_received:8227 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 主节点 - 5461-10922 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.0.89:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 主节点 - 10923-16383 (共 0 个槽位) abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.0.89:32441 从节点 937c7ef861bd0c1128f152b847059444b1dd5291 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 主节点 - 0-5460 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.10.203:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.9.66:31952) 负责 0 个槽位: 5461-10922 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.8.48:31411) 负责 0 个槽位: 10923-16383 节点 937c7ef861bd0c1128f152b847059444b1dd5291 (172.31.8.48:32651) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success check failover pod name failover pod name:rediscl-epxpxf-shard-58x-1 failover kill1 Success 6 check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale rediscl-epxpxf --auto-approve --force=true --components shard --cpu 200m --memory 0.6Gi --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-verticalscaling-28qgg created successfully, you can view the progress: kbcli cluster describe-ops rediscl-epxpxf-verticalscaling-28qgg -n ns-hbjkr check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-verticalscaling-28qgg ns-hbjkr VerticalScaling rediscl-epxpxf shard Running 0/6 May 28,2025 12:20 UTC+0800 check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Updating May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 12:21 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 12:20 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-1;secondary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406115603 11 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406115000 11 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406113592 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406115000 10 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406114598 9 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406115000 10 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:10 cluster_stats_messages_ping_sent:101 cluster_stats_messages_pong_sent:94 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:9 cluster_stats_messages_sent:205 cluster_stats_messages_ping_received:94 cluster_stats_messages_pong_received:100 cluster_stats_messages_fail_received:1 cluster_stats_messages_received:195 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406118000 10 connected 10923-16383 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406119104 10 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406118501 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406118099 11 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406120109 9 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406118000 11 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:10 cluster_stats_messages_ping_sent:152 cluster_stats_messages_pong_sent:154 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:1 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:330 cluster_stats_messages_ping_received:148 cluster_stats_messages_pong_received:150 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:309 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406123552 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406123553 10 connected 10923-16383 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406123000 11 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406123000 9 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406124556 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406123000 11 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:11 cluster_stats_messages_ping_sent:139 cluster_stats_messages_pong_sent:141 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:292 cluster_stats_messages_ping_received:135 cluster_stats_messages_pong_received:136 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:279 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406129000 11 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406129654 10 connected 10923-16383 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406128649 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406129151 9 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406129000 11 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406129553 9 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:11 cluster_stats_messages_ping_sent:74 cluster_stats_messages_pong_sent:53 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:128 cluster_stats_messages_ping_received:53 cluster_stats_messages_pong_received:75 cluster_stats_messages_received:128 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406133324 11 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406134329 11 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406133000 9 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406133827 9 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406133000 10 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406133525 10 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:9 cluster_stats_messages_ping_sent:135 cluster_stats_messages_pong_sent:106 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:247 cluster_stats_messages_ping_received:106 cluster_stats_messages_pong_received:134 cluster_stats_messages_received:240 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406138201 10 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406138000 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406139206 10 connected 10923-16383 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406137197 11 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406136000 9 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406139000 11 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:11 cluster_my_epoch:9 cluster_stats_messages_ping_sent:181 cluster_stats_messages_pong_sent:185 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_update_sent:1 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:391 cluster_stats_messages_ping_received:179 cluster_stats_messages_pong_received:179 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:369 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441 主节点 - 0-5460 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 从节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841 主节点 - 10923-16383 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752 主节点 - 5461-10922 (共 0 个槽位) 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.14.41:32441) 负责 0 个槽位: 0-5460 节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 (172.31.0.89:31841) 负责 0 个槽位: 10923-16383 节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 (172.31.10.203:30752) 负责 0 个槽位: 5461-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-verticalscaling-28qgg ns-hbjkr VerticalScaling rediscl-epxpxf shard Succeed 6/6 May 28,2025 12:20 UTC+0800 check ops status done ops_status:rediscl-epxpxf-verticalscaling-28qgg ns-hbjkr VerticalScaling rediscl-epxpxf shard Succeed 6/6 May 28,2025 12:20 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations rediscl-epxpxf-verticalscaling-28qgg --namespace ns-hbjkr ` opsrequest.operations.kubeblocks.io/rediscl-epxpxf-verticalscaling-28qgg patched `kbcli cluster delete-ops --name rediscl-epxpxf-verticalscaling-28qgg --force --auto-approve --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-verticalscaling-28qgg deleted test failover oom check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-rediscl-epxpxf" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-rediscl-epxpxf" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-rediscl-epxpxf namespace: ns-hbjkr spec: selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-0 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-rediscl-epxpxf.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-rediscl-epxpxf created apply test-chaos-mesh-oom-rediscl-epxpxf.yaml Success `rm -rf test-chaos-mesh-oom-rediscl-epxpxf.yaml` check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Updating May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 12:21 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 12:20 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1;secondary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748406257000 12 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406257874 12 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406258376 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406257367 10 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406256864 9 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406257000 10 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:10 cluster_stats_messages_ping_sent:352 cluster_stats_messages_pong_sent:358 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:9 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:721 cluster_stats_messages_ping_received:358 cluster_stats_messages_pong_received:351 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:712 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406261000 10 connected 10923-16383 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406261000 10 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406262866 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748406262061 12 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406262564 9 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406262000 12 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:10 cluster_stats_messages_ping_sent:405 cluster_stats_messages_pong_sent:414 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_update_sent:2 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:845 cluster_stats_messages_ping_received:408 cluster_stats_messages_pong_received:403 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:3 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:824 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406267445 10 connected 10923-16383 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406266000 12 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406266000 9 connected 5461-10922 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406265936 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748406266000 12 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406266438 10 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:12 cluster_stats_messages_ping_sent:64 cluster_stats_messages_pong_sent:43 cluster_stats_messages_sent:107 cluster_stats_messages_ping_received:43 cluster_stats_messages_pong_received:64 cluster_stats_messages_received:107 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406271000 12 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406272528 10 connected 10923-16383 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406272326 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406271000 9 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748406271824 12 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406271321 9 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:12 cluster_stats_messages_ping_sent:341 cluster_stats_messages_pong_sent:315 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:663 cluster_stats_messages_ping_received:310 cluster_stats_messages_pong_received:341 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:654 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748406276026 12 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406277533 12 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406276000 9 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406277533 9 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406277031 10 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406276000 10 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:9 cluster_stats_messages_ping_sent:393 cluster_stats_messages_pong_sent:380 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:780 cluster_stats_messages_ping_received:380 cluster_stats_messages_pong_received:391 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:773 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406282012 10 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406281000 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406282000 10 connected 10923-16383 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406282509 12 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406281000 9 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748406281000 12 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:9 cluster_stats_messages_ping_sent:432 cluster_stats_messages_pong_sent:455 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:20 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:3 cluster_stats_messages_update_sent:2 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:919 cluster_stats_messages_ping_received:449 cluster_stats_messages_pong_received:430 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:3 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:891 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441 从节点 937c7ef861bd0c1128f152b847059444b1dd5291 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651 主节点 - 0-5460 (共 0 个槽位) 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 从节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841 主节点 - 10923-16383 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752 主节点 - 5461-10922 (共 0 个槽位) 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 937c7ef861bd0c1128f152b847059444b1dd5291 (172.31.12.152:32651) 负责 0 个槽位: 0-5460 节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 (172.31.0.89:31841) 负责 0 个槽位: 10923-16383 节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 (172.31.10.203:30752) 负责 0 个槽位: 5461-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-rediscl-epxpxf" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-rediscl-epxpxf" not found check failover pod name failover pod name:rediscl-epxpxf-shard-58x-1 failover oom Success test failover dnsrandom check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-rediscl-epxpxf" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-rediscl-epxpxf" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnsrandom-rediscl-epxpxf namespace: ns-hbjkr spec: selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-1 mode: all action: random duration: 2m `kubectl apply -f test-chaos-mesh-dnsrandom-rediscl-epxpxf.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnsrandom-rediscl-epxpxf created apply test-chaos-mesh-dnsrandom-rediscl-epxpxf.yaml Success `rm -rf test-chaos-mesh-dnsrandom-rediscl-epxpxf.yaml` dnsrandom chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 12:21 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 12:20 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1;secondary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748406493572 12 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406494000 12 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406493572 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406494074 10 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406493000 9 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406493000 10 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:10 cluster_stats_messages_ping_sent:769 cluster_stats_messages_pong_sent:788 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:9 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:1568 cluster_stats_messages_ping_received:788 cluster_stats_messages_pong_received:768 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:1559 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406497000 10 connected 10923-16383 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406497088 10 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406498596 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748406497590 12 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406498000 9 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406498093 12 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:10 cluster_stats_messages_ping_sent:842 cluster_stats_messages_pong_sent:841 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_update_sent:2 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:1709 cluster_stats_messages_ping_received:835 cluster_stats_messages_pong_received:840 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:3 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:1688 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406503585 10 connected 10923-16383 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406503000 12 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406503000 9 connected 5461-10922 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406503000 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748406502000 12 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406502000 10 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:12 cluster_stats_messages_ping_sent:485 cluster_stats_messages_pong_sent:461 cluster_stats_messages_sent:946 cluster_stats_messages_ping_received:461 cluster_stats_messages_pong_received:485 cluster_stats_messages_received:946 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406506000 12 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406508551 10 connected 10923-16383 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406507547 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406507000 9 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748406507547 12 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406508000 9 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:12 cluster_stats_messages_ping_sent:762 cluster_stats_messages_pong_sent:741 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:1510 cluster_stats_messages_ping_received:736 cluster_stats_messages_pong_received:762 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:1501 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748406513165 12 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406511557 12 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406510000 9 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406512000 9 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406512160 10 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406511000 10 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:9 cluster_stats_messages_ping_sent:815 cluster_stats_messages_pong_sent:792 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:1614 cluster_stats_messages_ping_received:792 cluster_stats_messages_pong_received:813 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:1607 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406516313 10 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406517521 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406516000 10 connected 10923-16383 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406516000 12 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406516000 9 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748406517320 12 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:9 cluster_stats_messages_ping_sent:850 cluster_stats_messages_pong_sent:881 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:20 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:3 cluster_stats_messages_update_sent:2 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:1763 cluster_stats_messages_ping_received:875 cluster_stats_messages_pong_received:848 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:3 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:1735 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441 从节点 937c7ef861bd0c1128f152b847059444b1dd5291 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651 主节点 - 0-5460 (共 0 个槽位) 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 从节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841 主节点 - 10923-16383 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752 主节点 - 5461-10922 (共 0 个槽位) 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 937c7ef861bd0c1128f152b847059444b1dd5291 (172.31.12.152:32651) 负责 0 个槽位: 0-5460 节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 (172.31.0.89:31841) 负责 0 个槽位: 10923-16383 节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 (172.31.10.203:30752) 负责 0 个槽位: 5461-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-rediscl-epxpxf" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-rediscl-epxpxf" not found check failover pod name failover pod name:rediscl-epxpxf-shard-58x-1 failover dnsrandom Success test failover podfailure check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-rediscl-epxpxf" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-rediscl-epxpxf" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podfailure-rediscl-epxpxf namespace: ns-hbjkr spec: selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-1 mode: all action: pod-failure duration: 2m `kubectl apply -f test-chaos-mesh-podfailure-rediscl-epxpxf.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podfailure-rediscl-epxpxf created apply test-chaos-mesh-podfailure-rediscl-epxpxf.yaml Success `rm -rf test-chaos-mesh-podfailure-rediscl-epxpxf.yaml` podfailure chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Updating May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 12:21 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:20 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 12:20 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-1;secondary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406745000 13 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406744964 13 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406745065 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406744000 10 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406745969 9 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406745000 10 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:10 cluster_stats_messages_ping_sent:1193 cluster_stats_messages_pong_sent:1206 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:14 cluster_stats_messages_update_sent:2 cluster_stats_messages_sent:2416 cluster_stats_messages_ping_received:1206 cluster_stats_messages_pong_received:1192 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:2402 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406749000 10 connected 10923-16383 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406750563 10 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406750060 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406750000 13 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406749558 9 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406750563 13 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:10 cluster_stats_messages_ping_sent:1265 cluster_stats_messages_pong_sent:1271 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:3 cluster_stats_messages_update_sent:3 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:2564 cluster_stats_messages_ping_received:1265 cluster_stats_messages_pong_received:1263 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:8 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:2543 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406755335 10 connected 10923-16383 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406753324 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406754530 9 connected 5461-10922 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406755000 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406753000 13 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406754530 10 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:13 cluster_stats_messages_ping_sent:917 cluster_stats_messages_pong_sent:892 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_sent:1815 cluster_stats_messages_ping_received:887 cluster_stats_messages_pong_received:916 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:1806 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406759000 13 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406759000 13 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406758000 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406759571 9 connected 5461-10922 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406759570 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406758000 10 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:13 cluster_stats_messages_ping_sent:82 cluster_stats_messages_pong_sent:57 cluster_stats_messages_sent:139 cluster_stats_messages_ping_received:57 cluster_stats_messages_pong_received:82 cluster_stats_messages_received:139 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406764000 13 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406764430 13 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406762000 9 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406763426 9 connected 5461-10922 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406764531 10 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406764531 10 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:9 cluster_stats_messages_ping_sent:1240 cluster_stats_messages_pong_sent:1229 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:2 cluster_stats_messages_sent:2477 cluster_stats_messages_ping_received:1229 cluster_stats_messages_pong_received:1238 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:2471 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406768542 10 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406768743 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406769548 10 connected 10923-16383 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406768542 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406768000 9 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406767737 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:13 cluster_my_epoch:9 cluster_stats_messages_ping_sent:1272 cluster_stats_messages_pong_sent:1323 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:20 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:4 cluster_stats_messages_update_sent:3 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:2629 cluster_stats_messages_ping_received:1317 cluster_stats_messages_pong_received:1270 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:4 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:2601 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.14.41:32441 主节点 - 0-5460 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.12.152:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.9.66:31952 从节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841 主节点 - 10923-16383 (共 0 个槽位) b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752 主节点 - 5461-10922 (共 0 个槽位) 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.8.48:31411 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.14.41:32441) 负责 0 个槽位: 0-5460 节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 (172.31.0.89:31841) 负责 0 个槽位: 10923-16383 节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 (172.31.10.203:30752) 负责 0 个槽位: 5461-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-rediscl-epxpxf" force deleted Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-rediscl-epxpxf" not found check failover pod name failover pod name:rediscl-epxpxf-shard-58x-0 failover podfailure Success cluster stop check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster stop rediscl-epxpxf --auto-approve --force=true --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-stop-frvwh created successfully, you can view the progress: kbcli cluster describe-ops rediscl-epxpxf-stop-frvwh -n ns-hbjkr check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-stop-frvwh ns-hbjkr Stop rediscl-epxpxf shard Running 0/6 May 28,2025 12:34 UTC+0800 check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Stopped May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Stopped check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME check pod status done check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-stop-frvwh ns-hbjkr Stop rediscl-epxpxf shard Succeed 6/6 May 28,2025 12:34 UTC+0800 check ops status done ops_status:rediscl-epxpxf-stop-frvwh ns-hbjkr Stop rediscl-epxpxf shard Succeed 6/6 May 28,2025 12:34 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations rediscl-epxpxf-stop-frvwh --namespace ns-hbjkr ` opsrequest.operations.kubeblocks.io/rediscl-epxpxf-stop-frvwh patched `kbcli cluster delete-ops --name rediscl-epxpxf-stop-frvwh --force --auto-approve --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-stop-frvwh deleted cluster start check cluster status before ops check cluster status done cluster_status:Stopped `kbcli cluster start rediscl-epxpxf --force=true --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-start-bb5v2 created successfully, you can view the progress: kbcli cluster describe-ops rediscl-epxpxf-start-bb5v2 -n ns-hbjkr check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-start-bb5v2 ns-hbjkr Start rediscl-epxpxf shard Running 0/6 May 28,2025 12:34 UTC+0800 check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Updating May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 12:34 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-1;secondary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406924000 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406926060 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406925054 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406925000 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406924551 13 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406924049 13 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:64 cluster_stats_messages_pong_sent:77 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:4 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:151 cluster_stats_messages_ping_received:77 cluster_stats_messages_pong_received:60 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:145 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406931042 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406930000 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406929033 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406930000 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406930540 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406929535 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:63 cluster_stats_messages_pong_sent:56 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:130 cluster_stats_messages_ping_received:54 cluster_stats_messages_pong_received:62 cluster_stats_messages_meet_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:119 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406935582 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406935080 9 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406935783 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406935281 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406934275 9 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406935000 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:77 cluster_stats_messages_pong_sent:99 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:8 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:188 cluster_stats_messages_ping_received:97 cluster_stats_messages_pong_received:74 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:180 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406939576 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406940000 10 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406939778 10 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406938773 9 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406938000 13 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406939000 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:83 cluster_stats_messages_pong_sent:57 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:141 cluster_stats_messages_ping_received:57 cluster_stats_messages_pong_received:82 cluster_stats_messages_received:139 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406944000 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406944000 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406944428 13 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406944931 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406945433 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748406943522 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:101 cluster_stats_messages_pong_sent:111 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:218 cluster_stats_messages_ping_received:111 cluster_stats_messages_pong_received:97 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:217 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748406950060 13 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748406949053 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748406948000 9 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748406949053 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748406949556 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748406949000 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:104 cluster_stats_messages_pong_sent:80 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:194 cluster_stats_messages_ping_received:78 cluster_stats_messages_pong_received:102 cluster_stats_messages_meet_received:2 cluster_stats_messages_received:182 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752 主节点 - 5461-10922 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841 主节点 - 10923-16383 (共 0 个槽位) 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 从节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441 主节点 - 0-5460 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 (172.31.10.203:30752) 负责 0 个槽位: 5461-10922 节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 (172.31.12.152:31841) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.9.66:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-start-bb5v2 ns-hbjkr Start rediscl-epxpxf shard Succeed 6/6 May 28,2025 12:34 UTC+0800 check ops status done ops_status:rediscl-epxpxf-start-bb5v2 ns-hbjkr Start rediscl-epxpxf shard Succeed 6/6 May 28,2025 12:34 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations rediscl-epxpxf-start-bb5v2 --namespace ns-hbjkr ` opsrequest.operations.kubeblocks.io/rediscl-epxpxf-start-bb5v2 patched `kbcli cluster delete-ops --name rediscl-epxpxf-start-bb5v2 --force --auto-approve --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-start-bb5v2 deleted test failover timeoffset check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-rediscl-epxpxf" not found Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-rediscl-epxpxf" not found apiVersion: chaos-mesh.org/v1alpha1 kind: TimeChaos metadata: name: test-chaos-mesh-timeoffset-rediscl-epxpxf namespace: ns-hbjkr spec: selector: namespaces: - ns-hbjkr labelSelectors: apps.kubeblocks.io/pod-name: rediscl-epxpxf-shard-58x-0 mode: all timeOffset: '-10m' clockIds: - CLOCK_REALTIME duration: 2m `kubectl apply -f test-chaos-mesh-timeoffset-rediscl-epxpxf.yaml` timechaos.chaos-mesh.org/test-chaos-mesh-timeoffset-rediscl-epxpxf created apply test-chaos-mesh-timeoffset-rediscl-epxpxf.yaml Success `rm -rf test-chaos-mesh-timeoffset-rediscl-epxpxf.yaml` timeoffset chaos test waiting 120 seconds check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 12:34 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-1;secondary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407153274 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407152571 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407152269 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407153000 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407153000 13 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407151263 13 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:483 cluster_stats_messages_pong_sent:491 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:4 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:984 cluster_stats_messages_ping_received:491 cluster_stats_messages_pong_received:479 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:978 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407156610 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407156000 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407156000 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407155000 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407157616 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407156107 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:481 cluster_stats_messages_pong_sent:431 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:923 cluster_stats_messages_ping_received:429 cluster_stats_messages_pong_received:480 cluster_stats_messages_meet_received:2 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:912 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407162000 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407162526 9 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407161522 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407162827 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407162000 9 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407161000 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:465 cluster_stats_messages_pong_sent:519 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:8 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:996 cluster_stats_messages_ping_received:517 cluster_stats_messages_pong_received:462 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:988 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407166557 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407166000 10 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407167561 10 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407167059 9 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407165000 13 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407166054 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:495 cluster_stats_messages_pong_sent:494 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:990 cluster_stats_messages_ping_received:494 cluster_stats_messages_pong_received:494 cluster_stats_messages_received:988 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407170625 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407169000 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407172537 13 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407171733 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407171000 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407171000 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:527 cluster_stats_messages_pong_sent:544 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1077 cluster_stats_messages_ping_received:544 cluster_stats_messages_pong_received:523 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:1076 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407176787 13 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407175280 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407176585 9 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407176585 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407176284 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407175000 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:534 cluster_stats_messages_pong_sent:494 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:1038 cluster_stats_messages_ping_received:492 cluster_stats_messages_pong_received:532 cluster_stats_messages_meet_received:2 cluster_stats_messages_received:1026 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752 主节点 - 5461-10922 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841 主节点 - 10923-16383 (共 0 个槽位) 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 从节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441 主节点 - 0-5460 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 (172.31.10.203:30752) 负责 0 个槽位: 5461-10922 节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 (172.31.12.152:31841) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.9.66:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-rediscl-epxpxf --namespace ns-hbjkr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-rediscl-epxpxf" force deleted Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-rediscl-epxpxf" not found check failover pod name failover pod name:rediscl-epxpxf-shard-58x-0 failover timeoffset Success cluster shard scale-out check cluster status before ops check cluster status done cluster_status:Running No resources found in rediscl-epxpxf namespace. `kbcli cluster scale-out rediscl-epxpxf --auto-approve --force=true --components shard --replicas 1 --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-horizontalscaling-z7hg9 created successfully, you can view the progress: kbcli cluster describe-ops rediscl-epxpxf-horizontalscaling-z7hg9 -n ns-hbjkr check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-horizontalscaling-z7hg9 ns-hbjkr HorizontalScaling rediscl-epxpxf shard Running 0/3 May 28,2025 12:40 UTC+0800 check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Updating May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-4sx-2 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-6-100.us-west-2.compute.internal/172.31.6.100 May 28,2025 12:40 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-58x-2 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-9-61.us-west-2.compute.internal/172.31.9.61 May 28,2025 12:40 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-kss-2 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-11-169.us-west-2.compute.internal/172.31.11.169 May 28,2025 12:40 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-1;secondary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-4sx-2 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-58x-2 rediscl-epxpxf-shard-kss-0 rediscl-epxpxf-shard-kss-2 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407305000 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407306553 10 connected 10923-16383 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407304544 10 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407306553 9 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407305047 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407306000 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407304542 13 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407306054 13 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:769 cluster_stats_messages_pong_sent:788 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:4 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1567 cluster_stats_messages_ping_received:788 cluster_stats_messages_pong_received:765 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:1561 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407310000 13 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407311097 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407310000 10 connected 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407310090 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407310389 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407309000 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407310000 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407310590 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:771 cluster_stats_messages_pong_sent:736 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:1518 cluster_stats_messages_ping_received:733 cluster_stats_messages_pong_received:770 cluster_stats_messages_meet_received:3 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:1507 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-2 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 2 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407315000 13 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407316075 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407315000 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407315572 9 connected 5461-10922 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407315000 10 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407315673 13 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407315572 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407315000 10 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:81 cluster_stats_messages_pong_sent:90 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:172 cluster_stats_messages_ping_received:90 cluster_stats_messages_pong_received:82 cluster_stats_messages_received:172 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407319000 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407320657 9 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407320558 13 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407319000 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407318646 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407320555 9 connected 5461-10922 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407319651 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407319000 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:773 cluster_stats_messages_pong_sent:844 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:8 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:1629 cluster_stats_messages_ping_received:841 cluster_stats_messages_pong_received:770 cluster_stats_messages_meet_received:3 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:1621 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407324943 9 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407323000 13 connected 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407323937 10 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407323535 10 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407323000 10 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407323538 9 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407324000 13 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407324542 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:797 cluster_stats_messages_pong_sent:826 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1624 cluster_stats_messages_ping_received:826 cluster_stats_messages_pong_received:796 cluster_stats_messages_received:1622 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-2 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 2 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407329000 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407329979 10 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407329000 13 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407328571 13 connected 0-5460 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407328000 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407329000 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407328069 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407329000 10 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:107 cluster_stats_messages_pong_sent:113 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:221 cluster_stats_messages_ping_received:113 cluster_stats_messages_pong_received:108 cluster_stats_messages_received:221 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407334000 13 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407333000 10 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407332000 9 connected 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407333000 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407332513 13 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407334625 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407333620 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407333620 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:849 cluster_stats_messages_pong_sent:877 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:1732 cluster_stats_messages_ping_received:877 cluster_stats_messages_pong_received:845 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:1731 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407339202 13 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407337592 10 connected 10923-16383 ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407338000 13 connected 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407338000 10 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407339101 9 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407338195 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407338000 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407337000 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:861 cluster_stats_messages_pong_sent:846 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:1717 cluster_stats_messages_ping_received:843 cluster_stats_messages_pong_received:859 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:1705 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-2 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 2 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1f8fd6af7cc947dd36228a7f5048e8ca440f7feb 172.31.11.169:32489@31383,rediscl-epxpxf-shard-kss-2.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 0 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196 master - 0 1748407342565 13 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805 slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407343571 9 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230 master - 0 1748407342866 9 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358 slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407342866 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995 slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407343873 10 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665 slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407342565 13 connected 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415 slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407342565 10 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786 master - 0 1748407342000 10 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:174 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:175 cluster_stats_messages_pong_received:175 cluster_stats_messages_received:175 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 警告: 节点 rediscl-epxpxf-shard-kss-2 报告已知节点数为 '9'，与第一个节点报告的 '8' 不一致 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752 主节点 - 5461-10922 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841 主节点 - 10923-16383 (共 0 个槽位) 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 从节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441 主节点 - 0-5460 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-4sx-2 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-2 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-2 的拓扑视图... 警告: 节点 rediscl-epxpxf-shard-kss-2 看到 9 个节点，而参考节点看到 8 个节点 警告: 节点 rediscl-epxpxf-shard-kss-2 看到 1 个节点，它们在参考节点视图中不存在 额外的节点详情: ID IP地址 状态 ---------------------------------------- ------------- ---------------- 1f8fd6af7cc947dd36228a7f5048e8ca440f7feb 172.31.11.169:32489 从节点 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 (172.31.10.203:30752) 负责 0 个槽位: 5461-10922 节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 (172.31.12.152:31841) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.9.66:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 2 个从节点 ✅ 分片 58x: 1 个主节点, 2 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 5 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 否 - 所有节点看到一致的拓扑结构: 否 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 不健康 ❌ 检测到以下问题: - 集群视图不一致: 不同节点看到的集群大小或节点数量不同 - 拓扑结构不一致: 节点间对集群成员或角色的认知不一致 建议措施: 4. 使用 CLUSTER MEET 命令修复节点间的连接 5. 确保所有节点的配置(如cluster-announce-ip)正确 health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... health analyzing... ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407572000 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407572432 10 connected 10923-16383 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407570624 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407571530 13 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407572533 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407571000 10 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407571000 13 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407572533 13 connected 0-5460 1f8fd6af7cc947dd36228a7f5048e8ca440f7feb 172.31.11.169:32489@31383,rediscl-epxpxf-shard-kss-2.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407571000 9 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:1465 cluster_stats_messages_pong_sent:1490 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:4 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:2965 cluster_stats_messages_ping_received:1490 cluster_stats_messages_pong_received:1461 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:2959 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1f8fd6af7cc947dd36228a7f5048e8ca440f7feb 172.31.11.169:32489@31383,rediscl-epxpxf-shard-kss-2.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407576074 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407576000 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407575573 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407577779 9 connected 5461-10922 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407576000 10 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407577000 9 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407576575 13 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407577000 10 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407576575 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:1471 cluster_stats_messages_pong_sent:1434 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:2916 cluster_stats_messages_ping_received:1430 cluster_stats_messages_pong_received:1470 cluster_stats_messages_meet_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:2905 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-2 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 2 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407580259 13 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407581000 13 connected 0-5460 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407580000 10 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407581768 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407581000 9 connected 5461-10922 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407582000 9 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407582071 13 connected 1f8fd6af7cc947dd36228a7f5048e8ca440f7feb 172.31.11.169:32489@31383,rediscl-epxpxf-shard-kss-2.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407580763 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407582271 10 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:789 cluster_stats_messages_pong_sent:801 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1591 cluster_stats_messages_ping_received:801 cluster_stats_messages_pong_received:790 cluster_stats_messages_received:1591 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1f8fd6af7cc947dd36228a7f5048e8ca440f7feb 172.31.11.169:32489@31383,rediscl-epxpxf-shard-kss-2.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407585580 9 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407587083 9 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407585576 13 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407585579 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407586179 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407584000 13 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407587184 10 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407585174 9 connected 5461-10922 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407585576 10 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:1487 cluster_stats_messages_pong_sent:1566 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:8 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:3065 cluster_stats_messages_ping_received:1562 cluster_stats_messages_pong_received:1484 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:3057 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407590000 9 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407590515 13 connected 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407590518 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407589000 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407590000 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407591616 10 connected 10923-16383 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407591000 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407590612 13 connected 0-5460 1f8fd6af7cc947dd36228a7f5048e8ca440f7feb 172.31.11.169:32489@31383,rediscl-epxpxf-shard-kss-2.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407590616 9 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:1496 cluster_stats_messages_pong_sent:1537 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:3034 cluster_stats_messages_ping_received:1537 cluster_stats_messages_pong_received:1495 cluster_stats_messages_received:3032 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-2 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 2 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407596504 10 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407594000 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407595095 9 connected 5461-10922 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407595612 9 connected 1f8fd6af7cc947dd36228a7f5048e8ca440f7feb 172.31.11.169:32489@31383,rediscl-epxpxf-shard-kss-2.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407596001 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407596504 13 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407596000 13 connected 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407595000 10 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407595000 10 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:817 cluster_stats_messages_pong_sent:825 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:1643 cluster_stats_messages_ping_received:825 cluster_stats_messages_pong_received:818 cluster_stats_messages_received:1643 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407600000 9 connected 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407599582 10 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407600185 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407601193 13 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407599000 10 connected 10923-16383 ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407599180 13 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407599582 13 connected 0-5460 1f8fd6af7cc947dd36228a7f5048e8ca440f7feb 172.31.11.169:32489@31383,rediscl-epxpxf-shard-kss-2.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407599582 9 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407600000 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:1563 cluster_stats_messages_pong_sent:1594 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:3163 cluster_stats_messages_ping_received:1594 cluster_stats_messages_pong_received:1559 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:3162 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407604000 10 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407605000 13 connected 0-5460 1f8fd6af7cc947dd36228a7f5048e8ca440f7feb 172.31.11.169:32489@31383,rediscl-epxpxf-shard-kss-2.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407604562 9 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407604056 13 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407606068 9 connected 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407604559 10 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407605866 10 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407604000 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407605000 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:1577 cluster_stats_messages_pong_sent:1539 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:3126 cluster_stats_messages_ping_received:1535 cluster_stats_messages_pong_received:1575 cluster_stats_messages_meet_received:4 cluster_stats_messages_received:3114 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-2 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 2 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1f8fd6af7cc947dd36228a7f5048e8ca440f7feb 172.31.11.169:32489@31383,rediscl-epxpxf-shard-kss-2.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407607000 9 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407610529 13 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407610428 9 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407610000 9 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407610000 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407609020 10 connected ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777@31665,rediscl-epxpxf-shard-58x-2.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407610000 13 connected 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925@30415,rediscl-epxpxf-shard-4sx-2.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407609000 10 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407609000 10 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:9 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:865 cluster_stats_messages_pong_sent:713 cluster_stats_messages_meet_sent:4 cluster_stats_messages_sent:1582 cluster_stats_messages_ping_received:713 cluster_stats_messages_pong_received:869 cluster_stats_messages_received:1582 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752 主节点 - 5461-10922 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841 主节点 - 10923-16383 (共 0 个槽位) 023f1463a9b555cfc873c2e7f031623a70fe3735 172.31.6.100:30925 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 从节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 ac5f9cb943a88a89c2f87dec6ba9746fb2061a14 172.31.9.61:30777 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441 主节点 - 0-5460 (共 0 个槽位) 1f8fd6af7cc947dd36228a7f5048e8ca440f7feb 172.31.11.169:32489 从节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-4sx-2 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-2 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-2 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 (172.31.10.203:30752) 负责 0 个槽位: 5461-10922 节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 (172.31.12.152:31841) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.9.66:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 2 个从节点 ✅ 分片 58x: 1 个主节点, 2 个从节点 ✅ 分片 kss: 1 个主节点, 2 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 6 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success No resources found in rediscl-epxpxf namespace. check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-horizontalscaling-z7hg9 ns-hbjkr HorizontalScaling rediscl-epxpxf shard Succeed 3/3 May 28,2025 12:40 UTC+0800 check ops status done ops_status:rediscl-epxpxf-horizontalscaling-z7hg9 ns-hbjkr HorizontalScaling rediscl-epxpxf shard Succeed 3/3 May 28,2025 12:40 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations rediscl-epxpxf-horizontalscaling-z7hg9 --namespace ns-hbjkr ` opsrequest.operations.kubeblocks.io/rediscl-epxpxf-horizontalscaling-z7hg9 patched `kbcli cluster delete-ops --name rediscl-epxpxf-horizontalscaling-z7hg9 --force --auto-approve --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-horizontalscaling-z7hg9 deleted cluster shard scale-in check cluster status before ops check cluster status done cluster_status:Running No resources found in rediscl-epxpxf namespace. `kbcli cluster scale-in rediscl-epxpxf --auto-approve --force=true --components shard --replicas 1 --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-horizontalscaling-4phcw created successfully, you can view the progress: kbcli cluster describe-ops rediscl-epxpxf-horizontalscaling-4phcw -n ns-hbjkr check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-horizontalscaling-4phcw ns-hbjkr HorizontalScaling rediscl-epxpxf shard Running 0/3 May 28,2025 12:48 UTC+0800 check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:1Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 12:34 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-1;secondary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407707132 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407706128 10 connected 10923-16383 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407706000 13 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407707534 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407706000 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407708136 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:1806 cluster_stats_messages_pong_sent:1828 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:4 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:3644 cluster_stats_messages_ping_received:1828 cluster_stats_messages_pong_received:1802 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:3638 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407711000 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407711000 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407712439 9 connected 5461-10922 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407710431 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407709000 10 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407711435 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:1812 cluster_stats_messages_pong_sent:1777 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:3600 cluster_stats_messages_ping_received:1773 cluster_stats_messages_pong_received:1811 cluster_stats_messages_meet_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:3589 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407716000 9 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407716000 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407717521 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407717000 13 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407717521 10 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407716918 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:1813 cluster_stats_messages_pong_sent:1875 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:8 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:3700 cluster_stats_messages_ping_received:1871 cluster_stats_messages_pong_received:1810 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:3692 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407720000 9 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407719000 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407720881 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407720378 10 connected 10923-16383 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407720000 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407721384 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:1812 cluster_stats_messages_pong_sent:1854 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:3667 cluster_stats_messages_ping_received:1854 cluster_stats_messages_pong_received:1811 cluster_stats_messages_received:3665 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407724000 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407725000 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407725000 13 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407726100 10 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407725095 13 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407726503 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:1864 cluster_stats_messages_pong_sent:1898 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:3768 cluster_stats_messages_ping_received:1898 cluster_stats_messages_pong_received:1860 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:3767 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407730607 10 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407731000 13 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407731613 13 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407731511 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407731000 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407729000 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:1870 cluster_stats_messages_pong_sent:1835 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:3715 cluster_stats_messages_ping_received:1831 cluster_stats_messages_pong_received:1868 cluster_stats_messages_meet_received:4 cluster_stats_messages_received:3703 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752 主节点 - 5461-10922 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841 主节点 - 10923-16383 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 从节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441 主节点 - 0-5460 (共 0 个槽位) 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 (172.31.10.203:30752) 负责 0 个槽位: 5461-10922 节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 (172.31.12.152:31841) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.9.66:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success No resources found in rediscl-epxpxf namespace. check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-horizontalscaling-4phcw ns-hbjkr HorizontalScaling rediscl-epxpxf shard Succeed 3/3 May 28,2025 12:48 UTC+0800 check ops status done ops_status:rediscl-epxpxf-horizontalscaling-4phcw ns-hbjkr HorizontalScaling rediscl-epxpxf shard Succeed 3/3 May 28,2025 12:48 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations rediscl-epxpxf-horizontalscaling-4phcw --namespace ns-hbjkr ` opsrequest.operations.kubeblocks.io/rediscl-epxpxf-horizontalscaling-4phcw patched `kbcli cluster delete-ops --name rediscl-epxpxf-horizontalscaling-4phcw --force --auto-approve --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-horizontalscaling-4phcw deleted `kubectl get pvc -l app.kubernetes.io/instance=rediscl-epxpxf,apps.kubeblocks.io/component-name=shard,apps.kubeblocks.io/vct-name=data --namespace ns-hbjkr ` No resources found in ns-hbjkr namespace. rediscl-epxpxf shard data pvc is empty cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in rediscl-epxpxf namespace. `kbcli cluster volume-expand rediscl-epxpxf --auto-approve --force=true --components shard --volume-claim-templates data --storage 6Gi --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-volumeexpansion-mf6ws created successfully, you can view the progress: kbcli cluster describe-ops rediscl-epxpxf-volumeexpansion-mf6ws -n ns-hbjkr check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-volumeexpansion-mf6ws ns-hbjkr VolumeExpansion rediscl-epxpxf shard Running 0/6 May 28,2025 12:50 UTC+0800 check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Updating May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 12:34 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-1;secondary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407896105 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407895100 10 connected 10923-16383 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407894604 13 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407895000 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407893000 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407894095 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:2142 cluster_stats_messages_pong_sent:2167 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:4 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:4319 cluster_stats_messages_ping_received:2167 cluster_stats_messages_pong_received:2138 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:4313 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407899000 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407900000 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407899261 9 connected 5461-10922 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407899563 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407898000 10 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407900266 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:10 cluster_stats_messages_ping_sent:2156 cluster_stats_messages_pong_sent:2100 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:4267 cluster_stats_messages_ping_received:2096 cluster_stats_messages_pong_received:2155 cluster_stats_messages_meet_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:4256 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407905000 9 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407904000 13 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407905593 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407905000 13 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407905593 10 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407904000 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:2156 cluster_stats_messages_pong_sent:2207 cluster_stats_messages_meet_sent:2 cluster_stats_messages_fail_sent:8 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_sent:4375 cluster_stats_messages_ping_received:2203 cluster_stats_messages_pong_received:2153 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:4367 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407910000 9 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407910000 13 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407910507 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407910000 10 connected 10923-16383 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407910504 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407909497 13 connected 0-5460 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:13 cluster_stats_messages_ping_sent:2157 cluster_stats_messages_pong_sent:2195 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:4353 cluster_stats_messages_ping_received:2195 cluster_stats_messages_pong_received:2156 cluster_stats_messages_received:4351 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407914000 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407913000 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407913021 13 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407913220 10 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407913000 13 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748407914227 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:2208 cluster_stats_messages_pong_sent:2241 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:4455 cluster_stats_messages_ping_received:2241 cluster_stats_messages_pong_received:2204 cluster_stats_messages_fail_received:7 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:4454 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748407918691 10 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748407918590 13 connected 0-5460 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748407919096 13 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748407918000 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748407919698 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748407919000 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:15 cluster_my_epoch:9 cluster_stats_messages_ping_sent:2202 cluster_stats_messages_pong_sent:2183 cluster_stats_messages_fail_sent:10 cluster_stats_messages_sent:4395 cluster_stats_messages_ping_received:2179 cluster_stats_messages_pong_received:2200 cluster_stats_messages_meet_received:4 cluster_stats_messages_received:4383 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752 主节点 - 5461-10922 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841 主节点 - 10923-16383 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 从节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441 主节点 - 0-5460 (共 0 个槽位) 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 (172.31.10.203:30752) 负责 0 个槽位: 5461-10922 节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 (172.31.12.152:31841) 负责 0 个槽位: 10923-16383 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.9.66:32441) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success No resources found in rediscl-epxpxf namespace. check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-volumeexpansion-mf6ws ns-hbjkr VolumeExpansion rediscl-epxpxf shard Succeed 6/6 May 28,2025 12:50 UTC+0800 check ops status done ops_status:rediscl-epxpxf-volumeexpansion-mf6ws ns-hbjkr VolumeExpansion rediscl-epxpxf shard Succeed 6/6 May 28,2025 12:50 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations rediscl-epxpxf-volumeexpansion-mf6ws --namespace ns-hbjkr ` opsrequest.operations.kubeblocks.io/rediscl-epxpxf-volumeexpansion-mf6ws patched `kbcli cluster delete-ops --name rediscl-epxpxf-volumeexpansion-mf6ws --force --auto-approve --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-volumeexpansion-mf6ws deleted test failover drainnode check node drain check node drain success kubectl get pod rediscl-epxpxf-shard-58x-0 --namespace ns-hbjkr -o jsonpath='***.spec.nodeName***' get node name:ip-172-31-9-66.us-west-2.compute.internal success check if multiple pods are on the same node kubectl get pod rediscl-epxpxf-shard-58x-1 --namespace ns-hbjkr -o jsonpath='***.spec.nodeName***' get node name:ip-172-31-8-48.us-west-2.compute.internal success kubectl drain ip-172-31-9-66.us-west-2.compute.internal --delete-emptydir-data --ignore-daemonsets --force --grace-period 0 --timeout 60s node/ip-172-31-9-66.us-west-2.compute.internal cordoned Warning: ignoring DaemonSet-managed Pods: chaos-mesh/chaos-daemon-dqz4h, kube-system/aws-node-drggf, kube-system/ebs-csi-node-rftcq, kube-system/kube-proxy-k6bpb evicting pod ns-hbjkr/rediscl-epxpxf-shard-58x-0 pod/rediscl-epxpxf-shard-58x-0 evicted node/ip-172-31-9-66.us-west-2.compute.internal drained kubectl uncordon ip-172-31-9-66.us-west-2.compute.internal node/ip-172-31-9-66.us-west-2.compute.internal uncordoned check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-61.us-west-2.compute.internal/172.31.9.61 May 28,2025 12:53 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:34 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 12:34 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1;secondary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408067570 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408067970 10 connected 10923-16383 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408066966 16 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748408066000 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748408067000 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.61:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748408067000 16 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:16 cluster_my_epoch:10 cluster_stats_messages_ping_sent:2448 cluster_stats_messages_pong_sent:2482 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:9 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:4945 cluster_stats_messages_ping_received:2482 cluster_stats_messages_pong_received:2444 cluster_stats_messages_fail_received:6 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:4935 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748408071544 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408070540 16 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408071000 9 connected 5461-10922 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748408071042 9 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408072000 10 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.61:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748408072046 16 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:16 cluster_my_epoch:10 cluster_stats_messages_ping_sent:2458 cluster_stats_messages_pong_sent:2401 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:4871 cluster_stats_messages_ping_received:2397 cluster_stats_messages_pong_received:2457 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_received:4861 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408076028 9 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408077035 16 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748408076530 10 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408076028 10 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.61:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748408075000 16 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748408076000 9 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:16 cluster_my_epoch:16 cluster_stats_messages_ping_sent:124 cluster_stats_messages_pong_sent:92 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:217 cluster_stats_messages_ping_received:92 cluster_stats_messages_pong_received:125 cluster_stats_messages_received:217 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748408080507 9 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408081000 16 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408081000 9 connected 5461-10922 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408081009 10 connected 10923-16383 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748408081512 10 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.61:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748408081512 16 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:16 cluster_my_epoch:16 cluster_stats_messages_ping_sent:2468 cluster_stats_messages_pong_sent:2517 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:4991 cluster_stats_messages_ping_received:2511 cluster_stats_messages_pong_received:2466 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:4981 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748408084000 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748408086231 10 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408085000 16 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408086000 10 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.61:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748408086000 16 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408085527 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:16 cluster_my_epoch:9 cluster_stats_messages_ping_sent:2504 cluster_stats_messages_pong_sent:2559 cluster_stats_messages_meet_sent:1 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_sent:5069 cluster_stats_messages_ping_received:2559 cluster_stats_messages_pong_received:2500 cluster_stats_messages_fail_received:8 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:1 cluster_stats_messages_received:5070 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408090515 10 connected 10923-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.61:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave 937c7ef861bd0c1128f152b847059444b1dd5291 0 1748408089512 16 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408089810 16 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 0 1748408090816 9 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 1306bf11f70cd8cc27995470dddb8a4e1f847006 0 1748408090000 10 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408090000 9 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:16 cluster_my_epoch:9 cluster_stats_messages_ping_sent:2509 cluster_stats_messages_pong_sent:2490 cluster_stats_messages_fail_sent:10 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_sent:5010 cluster_stats_messages_ping_received:2486 cluster_stats_messages_pong_received:2507 cluster_stats_messages_meet_received:4 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:4999 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.10.203:30752 主节点 - 5461-10922 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.12.152:31841 主节点 - 10923-16383 (共 0 个槽位) 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 主节点 - 0-5460 (共 0 个槽位) 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 从节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.0.89:31411 从节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.61:32441 从节点 937c7ef861bd0c1128f152b847059444b1dd5291 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 (172.31.10.203:30752) 负责 0 个槽位: 5461-10922 节点 1306bf11f70cd8cc27995470dddb8a4e1f847006 (172.31.12.152:31841) 负责 0 个槽位: 10923-16383 节点 937c7ef861bd0c1128f152b847059444b1dd5291 (172.31.8.48:32651) 负责 0 个槽位: 0-5460 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success check failover pod name failover pod name:rediscl-epxpxf-shard-58x-1 failover drainnode Success cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart rediscl-epxpxf --auto-approve --force=true --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-restart-z58xj created successfully, you can view the progress: kbcli cluster describe-ops rediscl-epxpxf-restart-z58xj -n ns-hbjkr check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-restart-z58xj ns-hbjkr Restart rediscl-epxpxf shard Running 0/6 May 28,2025 12:56 UTC+0800 check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Updating May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-11-169.us-west-2.compute.internal/172.31.11.169 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:58 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-61.us-west-2.compute.internal/172.31.9.61 May 28,2025 12:56 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408358188 17 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408358000 19 connected 0-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408358589 18 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408359193 19 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408358000 18 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408358690 17 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:18 cluster_stats_messages_ping_sent:308 cluster_stats_messages_pong_sent:315 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:20 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:651 cluster_stats_messages_ping_received:309 cluster_stats_messages_pong_received:306 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:624 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408363052 17 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408362000 18 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408363000 19 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408364063 17 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408362000 19 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408363554 18 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:18 cluster_stats_messages_ping_sent:285 cluster_stats_messages_pong_sent:262 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:553 cluster_stats_messages_ping_received:262 cluster_stats_messages_pong_received:285 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:549 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408368148 17 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408368000 19 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408368550 17 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408368651 19 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408368000 18 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408367141 18 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:19 cluster_stats_messages_ping_sent:324 cluster_stats_messages_pong_sent:329 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:1 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:676 cluster_stats_messages_ping_received:323 cluster_stats_messages_pong_received:322 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:653 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408371000 19 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408372000 18 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408371549 17 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408372000 18 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408371000 19 connected 0-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408372859 17 connected 5461-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:19 cluster_stats_messages_ping_sent:76 cluster_stats_messages_pong_sent:46 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:123 cluster_stats_messages_ping_received:46 cluster_stats_messages_pong_received:77 cluster_stats_messages_received:123 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408377309 18 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408375000 17 connected 5461-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408377000 19 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408376000 19 connected 0-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408376000 17 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408377000 18 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:17 cluster_stats_messages_ping_sent:343 cluster_stats_messages_pong_sent:348 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:8 cluster_stats_messages_auth-req_sent:4 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:707 cluster_stats_messages_ping_received:342 cluster_stats_messages_pong_received:341 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:694 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408382507 18 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408382104 19 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408382507 17 connected 5461-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408381000 19 connected 0-5460 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408382205 18 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408380000 17 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:19 cluster_my_epoch:17 cluster_stats_messages_ping_sent:325 cluster_stats_messages_pong_sent:296 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_sent:627 cluster_stats_messages_ping_received:296 cluster_stats_messages_pong_received:325 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:623 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441 主节点 - 0-5460 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411 主节点 - 10923-16383 (共 0 个槽位) 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 主节点 - 5461-10922 (共 0 个槽位) 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.9.66:32441) 负责 0 个槽位: 0-5460 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.11.169:31411) 负责 0 个槽位: 10923-16383 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.14.41:31952) 负责 0 个槽位: 5461-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success check ops status `kbcli cluster list-ops rediscl-epxpxf --status all --namespace ns-hbjkr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME rediscl-epxpxf-restart-z58xj ns-hbjkr Restart rediscl-epxpxf shard Succeed 6/6 May 28,2025 12:56 UTC+0800 check ops status done ops_status:rediscl-epxpxf-restart-z58xj ns-hbjkr Restart rediscl-epxpxf shard Succeed 6/6 May 28,2025 12:56 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations rediscl-epxpxf-restart-z58xj --namespace ns-hbjkr ` opsrequest.operations.kubeblocks.io/rediscl-epxpxf-restart-z58xj patched `kbcli cluster delete-ops --name rediscl-epxpxf-restart-z58xj --force --auto-approve --namespace ns-hbjkr ` OpsRequest rediscl-epxpxf-restart-z58xj deleted patch rediscl-epxpxf shards 4 `kubectl patch cluster rediscl-epxpxf --namespace ns-hbjkr --type json -p '[***"op": "replace", "path": "/spec/shardings/0/shards", "value": '4'***]'` cluster.apps.kubeblocks.io/rediscl-epxpxf patched check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Creating May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-11-169.us-west-2.compute.internal/172.31.11.169 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:58 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-61.us-west-2.compute.internal/172.31.9.61 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-wrm-0 ns-hbjkr rediscl-epxpxf shard(shard-wrm) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-6-100.us-west-2.compute.internal/172.31.6.100 May 28,2025 13:00 UTC+0800 rediscl-epxpxf-shard-wrm-1 ns-hbjkr rediscl-epxpxf shard(shard-wrm) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 13:01 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0 rediscl-epxpxf-shard-wrm-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1 rediscl-epxpxf-shard-wrm-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408547326 17 connected 8681b7710417151d12f82b208dd7b216d490e190 172.31.12.152:32576@32098,rediscl-epxpxf-shard-wrm-1.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local slave f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 0 1748408547000 21 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408547000 19 connected 1365-5460 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408547024 18 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408546000 19 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408545000 18 connected 12288-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408548334 17 connected 6827-10922 f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 172.31.6.100:31063@30884,rediscl-epxpxf-shard-wrm-0.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local master - 0 1748408547000 21 connected 0-1364 5461-6826 10923-12287 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:21 cluster_my_epoch:18 cluster_stats_messages_ping_sent:673 cluster_stats_messages_pong_sent:674 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:20 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:4 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:1379 cluster_stats_messages_ping_received:667 cluster_stats_messages_pong_received:4767 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_update_received:1 cluster_stats_messages_received:5445 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 172.31.6.100:31063@30884,rediscl-epxpxf-shard-wrm-0.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local master - 0 1748408551000 21 connected 0-1364 5461-6826 10923-12287 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408551000 17 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408552000 18 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408552191 19 connected 1365-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408551588 17 connected 6827-10922 8681b7710417151d12f82b208dd7b216d490e190 172.31.12.152:32576@32098,rediscl-epxpxf-shard-wrm-1.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local slave f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 0 1748408552090 21 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408553195 19 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408552000 18 connected 12288-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:21 cluster_my_epoch:18 cluster_stats_messages_ping_sent:648 cluster_stats_messages_pong_sent:621 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:2 cluster_stats_messages_sent:1277 cluster_stats_messages_ping_received:621 cluster_stats_messages_pong_received:4744 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:14 cluster_stats_messages_received:5381 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408556585 17 connected 6827-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408557000 19 connected 1365-5460 8681b7710417151d12f82b208dd7b216d490e190 172.31.12.152:32576@32098,rediscl-epxpxf-shard-wrm-1.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local slave f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 0 1748408556786 21 connected b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408557590 17 connected f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 172.31.6.100:31063@30884,rediscl-epxpxf-shard-wrm-0.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local master - 0 1748408556284 21 connected 0-1364 5461-6826 10923-12287 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408555780 19 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408557000 18 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408557289 18 connected 12288-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:21 cluster_my_epoch:19 cluster_stats_messages_ping_sent:690 cluster_stats_messages_pong_sent:679 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:9 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:1400 cluster_stats_messages_ping_received:673 cluster_stats_messages_pong_received:4784 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:5465 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 8681b7710417151d12f82b208dd7b216d490e190 172.31.12.152:32576@32098,rediscl-epxpxf-shard-wrm-1.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local slave f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 0 1748408561096 21 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408560000 19 connected f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 172.31.6.100:31063@30884,rediscl-epxpxf-shard-wrm-0.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local master - 0 1748408561000 21 connected 0-1364 5461-6826 10923-12287 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408562000 18 connected 12288-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408560593 17 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408561000 18 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408562101 19 connected 1365-5460 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408562101 17 connected 6827-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:21 cluster_my_epoch:19 cluster_stats_messages_ping_sent:439 cluster_stats_messages_pong_sent:424 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:5 cluster_stats_messages_sent:869 cluster_stats_messages_ping_received:424 cluster_stats_messages_pong_received:4536 cluster_stats_messages_update_received:20 cluster_stats_messages_received:4980 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 172.31.6.100:31063@30884,rediscl-epxpxf-shard-wrm-0.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local master - 0 1748408565000 21 connected 0-1364 5461-6826 10923-12287 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408566000 18 connected 8681b7710417151d12f82b208dd7b216d490e190 172.31.12.152:32576@32098,rediscl-epxpxf-shard-wrm-1.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local slave f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 0 1748408566000 21 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408564000 17 connected 6827-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408565577 19 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408566684 19 connected 1365-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408565578 17 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408565679 18 connected 12288-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:21 cluster_my_epoch:17 cluster_stats_messages_ping_sent:720 cluster_stats_messages_pong_sent:731 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:8 cluster_stats_messages_auth-req_sent:4 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_update_sent:8 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:1475 cluster_stats_messages_ping_received:725 cluster_stats_messages_pong_received:4814 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:5550 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408571097 18 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408570000 19 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408571402 17 connected 6827-10922 f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 172.31.6.100:31063@30884,rediscl-epxpxf-shard-wrm-0.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local master - 0 1748408570000 21 connected 0-1364 5461-6826 10923-12287 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408570395 19 connected 1365-5460 8681b7710417151d12f82b208dd7b216d490e190 172.31.12.152:32576@32098,rediscl-epxpxf-shard-wrm-1.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local slave f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 0 1748408571097 21 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408571904 18 connected 12288-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408570000 17 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:21 cluster_my_epoch:17 cluster_stats_messages_ping_sent:705 cluster_stats_messages_pong_sent:682 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:2 cluster_stats_messages_sent:1395 cluster_stats_messages_ping_received:682 cluster_stats_messages_pong_received:4801 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:13 cluster_stats_messages_received:5498 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-wrm-0 ========== 检测到基础名称: rediscl-epxpxf-shard-wrm, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-wrm-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 8681b7710417151d12f82b208dd7b216d490e190 172.31.12.152:32576@32098,rediscl-epxpxf-shard-wrm-1.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local slave f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 0 1748408575516 21 connected f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 172.31.6.100:31063@30884,rediscl-epxpxf-shard-wrm-0.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408575000 21 connected 0-1364 5461-6826 10923-12287 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408576120 18 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408574307 17 connected 6827-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408575818 19 connected 1365-5460 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408574811 17 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408576321 18 connected 12288-16383 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408575315 19 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:21 cluster_my_epoch:21 cluster_stats_messages_ping_sent:181 cluster_stats_messages_pong_sent:28839 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:14 cluster_stats_messages_sent:29035 cluster_stats_messages_ping_received:166 cluster_stats_messages_pong_received:182 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:349 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-wrm-1 ========== 检测到基础名称: rediscl-epxpxf-shard-wrm, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-wrm-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408580000 19 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408580000 17 connected 6827-10922 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408579587 17 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408581096 18 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408579587 18 connected 12288-16383 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408580000 19 connected 1365-5460 8681b7710417151d12f82b208dd7b216d490e190 172.31.12.152:32576@32098,rediscl-epxpxf-shard-wrm-1.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local myself,slave f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 0 1748408578000 21 connected f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 172.31.6.100:31063@30884,rediscl-epxpxf-shard-wrm-0.rediscl-epxpxf-shard-wrm-headless.ns-hbjkr.svc.cluster.local master - 0 1748408580594 21 connected 0-1364 5461-6826 10923-12287 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:8 cluster_size:4 cluster_current_epoch:21 cluster_my_epoch:21 cluster_stats_messages_ping_sent:165 cluster_stats_messages_pong_sent:174 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:5 cluster_stats_messages_sent:345 cluster_stats_messages_ping_received:174 cluster_stats_messages_pong_received:4262 cluster_stats_messages_received:4436 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss - wrm 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 8681b7710417151d12f82b208dd7b216d490e190 172.31.12.152:32576 从节点 f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441 主节点 - 1365-5460 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411 主节点 - 12288-16383 (共 0 个槽位) 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 主节点 - 6827-10922 (共 0 个槽位) f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 172.31.6.100:31063 主节点 - 0-1364 5461-6826 10923-12287 (共 0 个槽位) 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-wrm-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-wrm-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.9.66:32441) 负责 0 个槽位: 1365-5460 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.11.169:31411) 负责 0 个槽位: 12288-16383 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.14.41:31952) 负责 0 个槽位: 6827-10922 节点 f56c1bf1b2fa3a1e8af19e8881d77917f7d20753 (172.31.6.100:31063) 负责 0 个槽位: 0-1364 5461-6826 10923-12287 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 分片 wrm: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 4 个主节点和 4 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success No resources found in ns-hbjkr namespace. job pod status: No resources found in ns-hbjkr namespace. job pod status: No resources found in ns-hbjkr namespace. job pod status: No resources found in ns-hbjkr namespace. check rediscl-epxpxf post-provision skip patch rediscl-epxpxf shards 3 `kubectl patch cluster rediscl-epxpxf --namespace ns-hbjkr --type json -p '[***"op": "replace", "path": "/spec/shardings/0/shards", "value": '3'***]'` cluster.apps.kubeblocks.io/rediscl-epxpxf patched check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Creating May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-11-169.us-west-2.compute.internal/172.31.11.169 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:58 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-61.us-west-2.compute.internal/172.31.9.61 May 28,2025 12:56 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408708335 22 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408709341 23 connected 1365-5460 5462-6826 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408707532 24 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408709542 23 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408708000 24 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408709000 22 connected 0-1364 5461 6827-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:24 cluster_stats_messages_ping_sent:1044 cluster_stats_messages_pong_sent:10592 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:20 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:14 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:11678 cluster_stats_messages_ping_received:1030 cluster_stats_messages_pong_received:7869 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_update_received:1 cluster_stats_messages_received:8910 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408713189 22 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408712000 24 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408714000 23 connected 1365-5460 5462-6826 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408713189 22 connected 0-1364 5461 6827-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408714093 23 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408714194 24 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:24 cluster_stats_messages_ping_sent:1002 cluster_stats_messages_pong_sent:984 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:8 cluster_stats_messages_sent:2000 cluster_stats_messages_ping_received:984 cluster_stats_messages_pong_received:9194 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:14 cluster_stats_messages_received:10194 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408718000 22 connected 0-1364 5461 6827-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408718000 23 connected 1365-5460 5462-6826 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408717000 22 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408718826 23 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408718323 24 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408717317 24 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:23 cluster_stats_messages_ping_sent:1058 cluster_stats_messages_pong_sent:10598 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:19 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:11697 cluster_stats_messages_ping_received:1037 cluster_stats_messages_pong_received:7883 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:8928 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408723000 23 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408722000 24 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408723538 22 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408723136 24 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408722633 23 connected 1365-5460 5462-6826 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408722532 22 connected 0-1364 5461 6827-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:23 cluster_stats_messages_ping_sent:797 cluster_stats_messages_pong_sent:782 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:10 cluster_stats_messages_sent:1590 cluster_stats_messages_ping_received:782 cluster_stats_messages_pong_received:8990 cluster_stats_messages_update_received:20 cluster_stats_messages_received:9792 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408727853 24 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408727000 22 connected 0-1364 5461 6827-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408727000 23 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408727000 23 connected 1365-5460 5462-6826 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408728000 22 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408728557 24 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:22 cluster_stats_messages_ping_sent:1083 cluster_stats_messages_pong_sent:10648 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:8 cluster_stats_messages_auth-req_sent:4 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_update_sent:18 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:11765 cluster_stats_messages_ping_received:1080 cluster_stats_messages_pong_received:7907 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:8998 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408731435 24 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408731937 23 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408733000 22 connected 0-1364 5461 6827-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408733446 23 connected 1365-5460 5462-6826 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408731000 24 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408732000 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:22 cluster_stats_messages_ping_sent:1049 cluster_stats_messages_pong_sent:1034 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:9 cluster_stats_messages_sent:2098 cluster_stats_messages_ping_received:1034 cluster_stats_messages_pong_received:9241 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:13 cluster_stats_messages_received:10290 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441 主节点 - 1365-5460 5462-6826 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411 主节点 - 10923-16383 (共 0 个槽位) 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 主节点 - 0-1364 5461 6827-10922 (共 0 个槽位) 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.9.66:32441) 负责 0 个槽位: 1365-5460 5462-6826 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.11.169:31411) 负责 0 个槽位: 10923-16383 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.14.41:31952) 负责 0 个槽位: 0-1364 5461 6827-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success No resources found in ns-hbjkr namespace. job pod status: No resources found in ns-hbjkr namespace. job pod status: No resources found in ns-hbjkr namespace. job pod status: No resources found in ns-hbjkr namespace. check rediscl-epxpxf pre-terminate skip get shard component name set shard component name:shard-58x cluster update terminationPolicy WipeOut `kbcli cluster update rediscl-epxpxf --termination-policy=WipeOut --namespace ns-hbjkr ` cluster.apps.kubeblocks.io/rediscl-epxpxf updated (no change) check cluster status `kbcli cluster list rediscl-epxpxf --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf ns-hbjkr WipeOut Running May 28,2025 11:38 UTC+0800 app.kubernetes.io/instance=rediscl-epxpxf check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-shard-4sx-0 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-11-169.us-west-2.compute.internal/172.31.11.169 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-4sx-1 ns-hbjkr rediscl-epxpxf shard(shard-4sx) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-58x-0 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-58x-1 ns-hbjkr rediscl-epxpxf shard(shard-58x) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 12:58 UTC+0800 rediscl-epxpxf-shard-kss-0 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-14-41.us-west-2.compute.internal/172.31.14.41 May 28,2025 12:56 UTC+0800 rediscl-epxpxf-shard-kss-1 ns-hbjkr rediscl-epxpxf shard(shard-kss) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-61.us-west-2.compute.internal/172.31.9.61 May 28,2025 12:56 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-shard-4sx-0 rediscl-epxpxf-shard-58x-0 rediscl-epxpxf-shard-kss-0;secondary: rediscl-epxpxf-shard-4sx-1 rediscl-epxpxf-shard-58x-1 rediscl-epxpxf-shard-kss-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408841105 22 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408840099 23 connected 1365-5460 5462-6826 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408841000 24 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408841608 23 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408840000 24 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408841000 22 connected 0-1364 5461 6827-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:24 cluster_stats_messages_ping_sent:1282 cluster_stats_messages_pong_sent:10825 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:20 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:14 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:12149 cluster_stats_messages_ping_received:1263 cluster_stats_messages_pong_received:8107 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_update_received:1 cluster_stats_messages_received:9381 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408846000 22 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408844000 24 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408845885 23 connected 1365-5460 5462-6826 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408845000 22 connected 0-1364 5461 6827-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408845000 23 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408846000 24 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:24 cluster_stats_messages_ping_sent:1238 cluster_stats_messages_pong_sent:1219 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:8 cluster_stats_messages_sent:2471 cluster_stats_messages_ping_received:1219 cluster_stats_messages_pong_received:9430 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:14 cluster_stats_messages_received:10665 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408850035 22 connected 0-1364 5461 6827-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408850000 23 connected 1365-5460 5462-6826 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408851042 22 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408850035 23 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408850000 24 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408849032 24 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:23 cluster_stats_messages_ping_sent:1300 cluster_stats_messages_pong_sent:10834 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:19 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:12175 cluster_stats_messages_ping_received:1273 cluster_stats_messages_pong_received:8125 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:9406 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408855000 23 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408855856 24 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408856057 22 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408855553 24 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408854850 23 connected 1365-5460 5462-6826 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408854548 22 connected 0-1364 5461 6827-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:23 cluster_stats_messages_ping_sent:1036 cluster_stats_messages_pong_sent:1033 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:10 cluster_stats_messages_sent:2080 cluster_stats_messages_ping_received:1033 cluster_stats_messages_pong_received:9229 cluster_stats_messages_update_received:20 cluster_stats_messages_received:10282 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408859719 24 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748408859000 22 connected 0-1364 5461 6827-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408860725 23 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408859000 23 connected 1365-5460 5462-6826 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408859000 22 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408860000 24 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:22 cluster_stats_messages_ping_sent:1320 cluster_stats_messages_pong_sent:10888 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:8 cluster_stats_messages_auth-req_sent:4 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_update_sent:18 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:12242 cluster_stats_messages_ping_received:1320 cluster_stats_messages_pong_received:8144 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:9475 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748408865106 24 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748408865000 23 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748408865509 22 connected 0-1364 5461 6827-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748408864000 23 connected 1365-5460 5462-6826 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748408865207 24 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748408863000 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:22 cluster_stats_messages_ping_sent:1297 cluster_stats_messages_pong_sent:1287 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:9 cluster_stats_messages_sent:2599 cluster_stats_messages_ping_received:1287 cluster_stats_messages_pong_received:9489 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:13 cluster_stats_messages_received:10791 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441 主节点 - 1365-5460 5462-6826 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411 主节点 - 10923-16383 (共 0 个槽位) 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 主节点 - 0-1364 5461 6827-10922 (共 0 个槽位) 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.9.66:32441) 负责 0 个槽位: 1365-5460 5462-6826 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.11.169:31411) 负责 0 个槽位: 10923-16383 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.14.41:31952) 负责 0 个槽位: 0-1364 5461 6827-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kubectl get backupschedule -l app.kubernetes.io/instance=rediscl-epxpxf ` `kubectl get backupschedule rediscl-epxpxf-shard-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/rediscl-epxpxf-shard-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/rediscl-epxpxf-shard-backup-schedule patched check backup status `kbcli cluster list-backups rediscl-epxpxf --namespace ns-hbjkr ` No backups found in ns-hbjkr namespace. No backups found in ns-hbjkr namespace. backup_status:-- No backups found in ns-hbjkr namespace. backup_status:-- `kubectl get backupschedule -l app.kubernetes.io/instance=rediscl-epxpxf ` `kubectl get backupschedule rediscl-epxpxf-shard-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/rediscl-epxpxf-shard-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/rediscl-epxpxf-shard-backup-schedule patched backup_status:rediscl-epxpxf-datafile-Running check backup status done backup_status:rediscl-epxpxf-datafile-20250528050900 ns-hbjkr rediscl-epxpxf datafile Completed 9045 12s Delete May 28,2025 13:09 UTC+0800 May 28,2025 13:09 UTC+0800 Jun 04,2025 13:09 UTC+0800 `kubectl get backupschedule -l app.kubernetes.io/instance=rediscl-epxpxf ` `kubectl get backupschedule rediscl-epxpxf-shard-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/rediscl-epxpxf-shard-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/rediscl-epxpxf-shard-backup-schedule patched (no change) cluster restore backup Error from server (NotFound): opsrequests.operations.kubeblocks.io "rediscl-epxpxf-backup" not found `kbcli cluster describe-backup --names rediscl-epxpxf-datafile-20250528050900 --namespace ns-hbjkr ` Name: rediscl-epxpxf-datafile-20250528050900 Cluster: rediscl-epxpxf Namespace: ns-hbjkr Spec: Method: datafile Policy Name: rediscl-epxpxf-shard-backup-policy Actions: dp-backup-shard-kss-0: ActionType: Job WorkloadName: dp-backup-shard-kss-0-rediscl-epxpxf-datafile-20250528050900-f1 TargetPodName: rediscl-epxpxf-shard-kss-1 Phase: Completed Start Time: May 28,2025 13:09 UTC+0800 Completion Time: May 28,2025 13:09 UTC+0800 dp-backup-shard-4sx-0: ActionType: Job WorkloadName: dp-backup-shard-4sx-0-rediscl-epxpxf-datafile-20250528050900-f1 TargetPodName: rediscl-epxpxf-shard-4sx-1 Phase: Completed Start Time: May 28,2025 13:09 UTC+0800 Completion Time: May 28,2025 13:09 UTC+0800 dp-backup-shard-58x-0: ActionType: Job WorkloadName: dp-backup-shard-58x-0-rediscl-epxpxf-datafile-20250528050900-f1 TargetPodName: rediscl-epxpxf-shard-58x-1 Phase: Completed Start Time: May 28,2025 13:09 UTC+0800 Completion Time: May 28,2025 13:09 UTC+0800 Status: Phase: Completed Total Size: 9045 ActionSet Name: redis-cluster-br Repository: backuprepo-kbcli-test Duration: 12s Expiration Time: Jun 04,2025 13:09 UTC+0800 Start Time: May 28,2025 13:09 UTC+0800 Completion Time: May 28,2025 13:09 UTC+0800 Path: /ns-hbjkr/rediscl-epxpxf-6ae30e0c-221c-43f1-8de6-046f4028dcbc/shard/rediscl-epxpxf-datafile-20250528050900 Warning Events: `kbcli cluster restore rediscl-epxpxf-backup --backup rediscl-epxpxf-datafile-20250528050900 --namespace ns-hbjkr ` Cluster rediscl-epxpxf-backup created check cluster status `kbcli cluster list rediscl-epxpxf-backup --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf-backup ns-hbjkr WipeOut Creating May 28,2025 13:09 UTC+0800 cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf-backup --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-backup-shard-2k6-0 ns-hbjkr rediscl-epxpxf-backup shard(shard-2k6) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 13:09 UTC+0800 rediscl-epxpxf-backup-shard-2k6-1 ns-hbjkr rediscl-epxpxf-backup shard(shard-2k6) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-0-89.us-west-2.compute.internal/172.31.0.89 May 28,2025 13:10 UTC+0800 rediscl-epxpxf-backup-shard-kdv-0 ns-hbjkr rediscl-epxpxf-backup shard(shard-kdv) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 13:09 UTC+0800 rediscl-epxpxf-backup-shard-kdv-1 ns-hbjkr rediscl-epxpxf-backup shard(shard-kdv) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-61.us-west-2.compute.internal/172.31.9.61 May 28,2025 13:10 UTC+0800 rediscl-epxpxf-backup-shard-trt-0 ns-hbjkr rediscl-epxpxf-backup shard(shard-trt) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-6-100.us-west-2.compute.internal/172.31.6.100 May 28,2025 13:09 UTC+0800 rediscl-epxpxf-backup-shard-trt-1 ns-hbjkr rediscl-epxpxf-backup shard(shard-trt) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 13:10 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-backup-shard-2k6-0 rediscl-epxpxf-backup-shard-kdv-0 rediscl-epxpxf-backup-shard-trt-0;secondary: rediscl-epxpxf-backup-shard-2k6-1 rediscl-epxpxf-backup-shard-kdv-1 rediscl-epxpxf-backup-shard-trt-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748409072405 22 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748409071400 23 connected 1365-5460 5462-6826 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748409072000 24 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748409070394 23 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748409071000 24 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748409072506 22 connected 0-1364 5461 6827-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:24 cluster_stats_messages_ping_sent:1688 cluster_stats_messages_pong_sent:11248 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:20 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:14 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:12978 cluster_stats_messages_ping_received:1686 cluster_stats_messages_pong_received:8513 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_update_received:1 cluster_stats_messages_received:10210 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748409076000 22 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748409075000 24 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748409075533 23 connected 1365-5460 5462-6826 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748409076035 22 connected 0-1364 5461 6827-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748409077039 23 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748409075000 24 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:24 cluster_stats_messages_ping_sent:1644 cluster_stats_messages_pong_sent:1643 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:8 cluster_stats_messages_sent:3301 cluster_stats_messages_ping_received:1643 cluster_stats_messages_pong_received:9836 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:14 cluster_stats_messages_received:11495 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748409081271 22 connected 0-1364 5461 6827-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748409080000 23 connected 1365-5460 5462-6826 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748409081000 22 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748409080263 23 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748409081000 24 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748409080000 24 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:23 cluster_stats_messages_ping_sent:1715 cluster_stats_messages_pong_sent:11245 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:19 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:13001 cluster_stats_messages_ping_received:1684 cluster_stats_messages_pong_received:8540 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:10232 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748409085000 23 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748409085000 24 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748409086055 22 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748409086000 24 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748409086156 23 connected 1365-5460 5462-6826 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748409086559 22 connected 0-1364 5461 6827-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:23 cluster_stats_messages_ping_sent:1464 cluster_stats_messages_pong_sent:1437 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:10 cluster_stats_messages_sent:2912 cluster_stats_messages_ping_received:1437 cluster_stats_messages_pong_received:9657 cluster_stats_messages_update_received:20 cluster_stats_messages_received:11114 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748409090738 24 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748409091000 22 connected 0-1364 5461 6827-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748409091245 23 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748409090234 23 connected 1365-5460 5462-6826 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748409090000 22 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748409091000 24 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:22 cluster_stats_messages_ping_sent:1750 cluster_stats_messages_pong_sent:11307 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:8 cluster_stats_messages_auth-req_sent:4 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_update_sent:18 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:13091 cluster_stats_messages_ping_received:1739 cluster_stats_messages_pong_received:8574 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:10324 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748409095567 24 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748409094000 23 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748409095969 22 connected 0-1364 5461 6827-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748409094459 23 connected 1365-5460 5462-6826 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748409095465 24 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748409095000 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:22 cluster_stats_messages_ping_sent:1715 cluster_stats_messages_pong_sent:1708 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:9 cluster_stats_messages_sent:3438 cluster_stats_messages_ping_received:1708 cluster_stats_messages_pong_received:9907 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:13 cluster_stats_messages_received:11630 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441 主节点 - 1365-5460 5462-6826 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411 主节点 - 10923-16383 (共 0 个槽位) 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 主节点 - 0-1364 5461 6827-10922 (共 0 个槽位) 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.9.66:32441) 负责 0 个槽位: 1365-5460 5462-6826 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.11.169:31411) 负责 0 个槽位: 10923-16383 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.14.41:31952) 负责 0 个槽位: 0-1364 5461 6827-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kbcli cluster describe-backup --names rediscl-epxpxf-datafile-20250528050900 --namespace ns-hbjkr ` Name: rediscl-epxpxf-datafile-20250528050900 Cluster: rediscl-epxpxf Namespace: ns-hbjkr Spec: Method: datafile Policy Name: rediscl-epxpxf-shard-backup-policy Actions: dp-backup-shard-kss-0: ActionType: Job WorkloadName: dp-backup-shard-kss-0-rediscl-epxpxf-datafile-20250528050900-f1 TargetPodName: rediscl-epxpxf-shard-kss-1 Phase: Completed Start Time: May 28,2025 13:09 UTC+0800 Completion Time: May 28,2025 13:09 UTC+0800 dp-backup-shard-4sx-0: ActionType: Job WorkloadName: dp-backup-shard-4sx-0-rediscl-epxpxf-datafile-20250528050900-f1 TargetPodName: rediscl-epxpxf-shard-4sx-1 Phase: Completed Start Time: May 28,2025 13:09 UTC+0800 Completion Time: May 28,2025 13:09 UTC+0800 dp-backup-shard-58x-0: ActionType: Job WorkloadName: dp-backup-shard-58x-0-rediscl-epxpxf-datafile-20250528050900-f1 TargetPodName: rediscl-epxpxf-shard-58x-1 Phase: Completed Start Time: May 28,2025 13:09 UTC+0800 Completion Time: May 28,2025 13:09 UTC+0800 Status: Phase: Completed Total Size: 9045 ActionSet Name: redis-cluster-br Repository: backuprepo-kbcli-test Duration: 12s Expiration Time: Jun 04,2025 13:09 UTC+0800 Start Time: May 28,2025 13:09 UTC+0800 Completion Time: May 28,2025 13:09 UTC+0800 Path: /ns-hbjkr/rediscl-epxpxf-6ae30e0c-221c-43f1-8de6-046f4028dcbc/shard/rediscl-epxpxf-datafile-20250528050900 Warning Events: cluster connect kbcli unsupported engine type: shard-58x delete cluster rediscl-epxpxf-backup `kbcli cluster delete rediscl-epxpxf-backup --auto-approve --namespace ns-hbjkr ` Cluster rediscl-epxpxf-backup deleted pod_info:rediscl-epxpxf-backup-shard-2k6-0 3/3 Running 0 2m58s rediscl-epxpxf-backup-shard-2k6-1 3/3 Running 0 2m40s rediscl-epxpxf-backup-shard-kdv-0 3/3 Running 0 2m58s rediscl-epxpxf-backup-shard-kdv-1 3/3 Running 0 2m33s rediscl-epxpxf-backup-shard-trt-0 3/3 Running 0 2m58s rediscl-epxpxf-backup-shard-trt-1 3/3 Running 0 2m42s No resources found in ns-hbjkr namespace. delete cluster pod done No resources found in ns-hbjkr namespace. check cluster resource non-exist OK: pvc No resources found in ns-hbjkr namespace. delete cluster done No resources found in ns-hbjkr namespace. No resources found in ns-hbjkr namespace. No resources found in ns-hbjkr namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups rediscl-epxpxf-datafile-20250528050900 --namespace ns-hbjkr ` backup.dataprotection.kubeblocks.io/rediscl-epxpxf-datafile-20250528050900 patched `kbcli cluster delete-backup rediscl-epxpxf --name rediscl-epxpxf-datafile-20250528050900 --force --auto-approve --namespace ns-hbjkr ` Backup rediscl-epxpxf-datafile-20250528050900 deleted No opsrequests found in ns-hbjkr namespace. cluster datafile backup `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.name***"` `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.namespace***"` `kubectl get secrets kb-backuprepo-7xclf -n kb-ijhhq -o jsonpath="***.data.accessKeyId***"` `kubectl get secrets kb-backuprepo-7xclf -n kb-ijhhq -o jsonpath="***.data.secretAccessKey***"` KUBEBLOCKS NAMESPACE:kb-ijhhq get kubeblocks namespace done `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-ijhhq -o jsonpath="***.items[0].data.root-user***"` `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-ijhhq -o jsonpath="***.items[0].data.root-password***"` minio_user:kbclitest,minio_password:kbclitest,minio_endpoint:kbcli-test-minio.kb-ijhhq.svc.cluster.local:9000 list minio bucket kbcli-test `echo 'mc config host add minioserver http://kbcli-test-minio.kb-ijhhq.svc.cluster.local:9000 kbclitest kbclitest;mc ls minioserver' | kubectl exec -it kbcli-test-minio-6558dfb75f-pwqk2 --namespace kb-ijhhq -- bash` Unable to use a TTY - input is not a terminal or the right kind of file list minio bucket done default backuprepo:backuprepo-kbcli-test exists `kbcli cluster backup rediscl-epxpxf --method datafile --namespace ns-hbjkr ` Backup backup-ns-hbjkr-rediscl-epxpxf-20250528131323 created successfully, you can view the progress: kbcli cluster list-backups --name=backup-ns-hbjkr-rediscl-epxpxf-20250528131323 -n ns-hbjkr check backup status `kbcli cluster list-backups rediscl-epxpxf --namespace ns-hbjkr ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION DELETION-POLICY CREATE-TIME COMPLETION-TIME EXPIRATION backup-ns-hbjkr-rediscl-epxpxf-20250528131323 ns-hbjkr rediscl-epxpxf datafile Running Delete May 28,2025 13:13 UTC+0800 backup_status:rediscl-epxpxf-datafile-Running backup_status:rediscl-epxpxf-datafile-Running check backup status done backup_status:backup-ns-hbjkr-rediscl-epxpxf-20250528131323 ns-hbjkr rediscl-epxpxf datafile Completed 9048 11s Delete May 28,2025 13:13 UTC+0800 May 28,2025 13:13 UTC+0800 cluster restore backup Error from server (NotFound): opsrequests.operations.kubeblocks.io "rediscl-epxpxf-backup" not found `kbcli cluster describe-backup --names backup-ns-hbjkr-rediscl-epxpxf-20250528131323 --namespace ns-hbjkr ` Name: backup-ns-hbjkr-rediscl-epxpxf-20250528131323 Cluster: rediscl-epxpxf Namespace: ns-hbjkr Spec: Method: datafile Policy Name: rediscl-epxpxf-shard-backup-policy Actions: dp-backup-shard-kss-0: ActionType: Job WorkloadName: dp-backup-shard-kss-0-backup-ns-hbjkr-rediscl-epxpxf-2025052813 TargetPodName: rediscl-epxpxf-shard-kss-1 Phase: Completed Start Time: May 28,2025 13:13 UTC+0800 Completion Time: May 28,2025 13:13 UTC+0800 dp-backup-shard-4sx-0: ActionType: Job WorkloadName: dp-backup-shard-4sx-0-backup-ns-hbjkr-rediscl-epxpxf-2025052813 TargetPodName: rediscl-epxpxf-shard-4sx-1 Phase: Completed Start Time: May 28,2025 13:13 UTC+0800 Completion Time: May 28,2025 13:13 UTC+0800 dp-backup-shard-58x-0: ActionType: Job WorkloadName: dp-backup-shard-58x-0-backup-ns-hbjkr-rediscl-epxpxf-2025052813 TargetPodName: rediscl-epxpxf-shard-58x-1 Phase: Completed Start Time: May 28,2025 13:13 UTC+0800 Completion Time: May 28,2025 13:13 UTC+0800 Status: Phase: Completed Total Size: 9048 ActionSet Name: redis-cluster-br Repository: backuprepo-kbcli-test Duration: 11s Start Time: May 28,2025 13:13 UTC+0800 Completion Time: May 28,2025 13:13 UTC+0800 Path: /ns-hbjkr/rediscl-epxpxf-6ae30e0c-221c-43f1-8de6-046f4028dcbc/shard/backup-ns-hbjkr-rediscl-epxpxf-20250528131323 Warning Events: `kbcli cluster restore rediscl-epxpxf-backup --backup backup-ns-hbjkr-rediscl-epxpxf-20250528131323 --namespace ns-hbjkr ` Cluster rediscl-epxpxf-backup created check cluster status `kbcli cluster list rediscl-epxpxf-backup --show-labels --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS rediscl-epxpxf-backup ns-hbjkr WipeOut Creating May 28,2025 13:13 UTC+0800 cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances rediscl-epxpxf-backup --namespace ns-hbjkr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME rediscl-epxpxf-backup-shard-fzg-0 ns-hbjkr rediscl-epxpxf-backup shard(shard-fzg) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-10-203.us-west-2.compute.internal/172.31.10.203 May 28,2025 13:13 UTC+0800 rediscl-epxpxf-backup-shard-fzg-1 ns-hbjkr rediscl-epxpxf-backup shard(shard-fzg) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-8-48.us-west-2.compute.internal/172.31.8.48 May 28,2025 13:14 UTC+0800 rediscl-epxpxf-backup-shard-hdn-0 ns-hbjkr rediscl-epxpxf-backup shard(shard-hdn) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-12-152.us-west-2.compute.internal/172.31.12.152 May 28,2025 13:13 UTC+0800 rediscl-epxpxf-backup-shard-hdn-1 ns-hbjkr rediscl-epxpxf-backup shard(shard-hdn) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-61.us-west-2.compute.internal/172.31.9.61 May 28,2025 13:14 UTC+0800 rediscl-epxpxf-backup-shard-vdl-0 ns-hbjkr rediscl-epxpxf-backup shard(shard-vdl) Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-6-100.us-west-2.compute.internal/172.31.6.100 May 28,2025 13:13 UTC+0800 rediscl-epxpxf-backup-shard-vdl-1 ns-hbjkr rediscl-epxpxf-backup shard(shard-vdl) Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:6Gi ip-172-31-9-66.us-west-2.compute.internal/172.31.9.66 May 28,2025 13:14 UTC+0800 check pod status done check cluster role check cluster role done primary: rediscl-epxpxf-backup-shard-fzg-0 rediscl-epxpxf-backup-shard-hdn-0 rediscl-epxpxf-backup-shard-vdl-0;secondary: rediscl-epxpxf-backup-shard-fzg-1 rediscl-epxpxf-backup-shard-hdn-1 rediscl-epxpxf-backup-shard-vdl-1 analyze cluster health check cluster connect `` ========== 节点: rediscl-epxpxf-shard-4sx-0 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748409329857 22 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748409328546 23 connected 1365-5460 5462-6826 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748409328849 24 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748409329352 23 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748409329000 24 connected 10923-16383 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748409328000 22 connected 0-1364 5461 6827-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:24 cluster_stats_messages_ping_sent:2153 cluster_stats_messages_pong_sent:11696 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:20 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_auth-ack_sent:1 cluster_stats_messages_update_sent:14 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:13891 cluster_stats_messages_ping_received:2134 cluster_stats_messages_pong_received:8978 cluster_stats_messages_meet_received:2 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_update_received:1 cluster_stats_messages_received:11123 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-4sx-1 ========== 检测到基础名称: rediscl-epxpxf-shard-4sx, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-4sx-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748409333817 22 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local myself,slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748409333000 24 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748409334319 23 connected 1365-5460 5462-6826 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748409333315 22 connected 0-1364 5461 6827-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748409334000 23 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748409334000 24 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:24 cluster_stats_messages_ping_sent:2101 cluster_stats_messages_pong_sent:2104 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:8 cluster_stats_messages_sent:4219 cluster_stats_messages_ping_received:2104 cluster_stats_messages_pong_received:10293 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:14 cluster_stats_messages_received:12413 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-0 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748409339000 22 connected 0-1364 5461 6827-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748409339000 23 connected 1365-5460 5462-6826 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748409338525 22 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748409338625 23 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748409339128 24 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748409339631 24 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:23 cluster_stats_messages_ping_sent:2186 cluster_stats_messages_pong_sent:11717 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:15 cluster_stats_messages_auth-req_sent:5 cluster_stats_messages_update_sent:19 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:13944 cluster_stats_messages_ping_received:2156 cluster_stats_messages_pong_received:9011 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:4 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_received:11175 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-58x-1 ========== 检测到基础名称: rediscl-epxpxf-shard-58x, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-58x-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local myself,slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748409342000 23 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748409342578 24 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748409343583 22 connected 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748409343000 24 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748409343000 23 connected 1365-5460 5462-6826 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748409343583 22 connected 0-1364 5461 6827-10922 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:23 cluster_stats_messages_ping_sent:1927 cluster_stats_messages_pong_sent:1898 cluster_stats_messages_meet_sent:1 cluster_stats_messages_update_sent:10 cluster_stats_messages_sent:3836 cluster_stats_messages_ping_received:1898 cluster_stats_messages_pong_received:10120 cluster_stats_messages_update_received:20 cluster_stats_messages_received:12038 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-0 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 0 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748409348292 24 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,master - 0 1748409347000 22 connected 0-1364 5461 6827-10922 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748409347789 23 connected abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748409346784 23 connected 1365-5460 5462-6826 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748409347000 22 connected 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748409348795 24 connected 10923-16383 --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:22 cluster_stats_messages_ping_sent:2216 cluster_stats_messages_pong_sent:11782 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:8 cluster_stats_messages_auth-req_sent:4 cluster_stats_messages_auth-ack_sent:2 cluster_stats_messages_update_sent:18 cluster_stats_messages_mfstart_sent:1 cluster_stats_messages_sent:14032 cluster_stats_messages_ping_received:2214 cluster_stats_messages_pong_received:9040 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:5 cluster_stats_messages_auth-req_received:2 cluster_stats_messages_auth-ack_received:3 cluster_stats_messages_received:11265 total_cluster_links_buffer_limit_exceeded:0 ========== 节点: rediscl-epxpxf-shard-kss-1 ========== 检测到基础名称: rediscl-epxpxf-shard-kss, 分片编号: 1 使用 Secret: rediscl-epxpxf-shard-kss-account-default 密码获取成功，前缀: Xm... 使用Pod环境变量中的Redis端口: 6379 --- 集群节点信息 --- 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841@30786,rediscl-epxpxf-shard-4sx-1.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local slave 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 0 1748409352000 24 connected 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651@30358,rediscl-epxpxf-shard-58x-1.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local slave abe77a21de6c181788175f352f48ae6c8f30fe78 0 1748409352593 23 connected 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952@30805,rediscl-epxpxf-shard-kss-0.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local master - 0 1748409352593 22 connected 0-1364 5461 6827-10922 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441@32196,rediscl-epxpxf-shard-58x-0.rediscl-epxpxf-shard-58x-headless.ns-hbjkr.svc.cluster.local master - 0 1748409352896 23 connected 1365-5460 5462-6826 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411@31995,rediscl-epxpxf-shard-4sx-0.rediscl-epxpxf-shard-4sx-headless.ns-hbjkr.svc.cluster.local master - 0 1748409353599 24 connected 10923-16383 b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752@31230,rediscl-epxpxf-shard-kss-1.rediscl-epxpxf-shard-kss-headless.ns-hbjkr.svc.cluster.local myself,slave 3c95ff667aa1ff475357549da4fdd1a335207795 0 1748409351000 22 connected --- 集群状态信息 --- cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:24 cluster_my_epoch:22 cluster_stats_messages_ping_sent:2173 cluster_stats_messages_pong_sent:2170 cluster_stats_messages_meet_sent:1 cluster_stats_messages_fail_sent:5 cluster_stats_messages_update_sent:9 cluster_stats_messages_sent:4358 cluster_stats_messages_ping_received:2170 cluster_stats_messages_pong_received:10365 cluster_stats_messages_fail_received:1 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_update_received:13 cluster_stats_messages_received:12550 total_cluster_links_buffer_limit_exceeded:0 ========== 集群健康分析 ========== 使用节点 rediscl-epxpxf-shard-4sx-0 作为参考基准 检测到以下分片标识符: - 4sx - 58x - kss 参考节点观察到的拓扑结构: ID IP地址 状态 主节点ID 槽位 ---------------------------------------- ------------- ---------------- ---------------------------------------- -------------- b0b5e16a0abb7300e57f420b5d5c1e82bc14e355 172.31.9.61:30752 从节点 3c95ff667aa1ff475357549da4fdd1a335207795 abe77a21de6c181788175f352f48ae6c8f30fe78 172.31.9.66:32441 主节点 - 1365-5460 5462-6826 (共 0 个槽位) 1306bf11f70cd8cc27995470dddb8a4e1f847006 172.31.0.89:31841 从节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 937c7ef861bd0c1128f152b847059444b1dd5291 172.31.8.48:32651 从节点 abe77a21de6c181788175f352f48ae6c8f30fe78 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b 172.31.11.169:31411 主节点 - 10923-16383 (共 0 个槽位) 3c95ff667aa1ff475357549da4fdd1a335207795 172.31.14.41:31952 主节点 - 0-1364 5461 6827-10922 (共 0 个槽位) 正在分析节点 rediscl-epxpxf-shard-4sx-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-58x-1 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-0 的拓扑视图... 正在分析节点 rediscl-epxpxf-shard-kss-1 的拓扑视图... 槽位分配统计: - 总槽位数: 16384 - 未覆盖槽位数: 0 - 重复覆盖槽位数: 0 - 正确覆盖槽位数: 16384 主节点槽位分配明细: 节点 abe77a21de6c181788175f352f48ae6c8f30fe78 (172.31.9.66:32441) 负责 0 个槽位: 1365-5460 5462-6826 节点 4d5bdab6e5357bbe23cfb0ebb9b24c7bc91a0b0b (172.31.11.169:31411) 负责 0 个槽位: 10923-16383 节点 3c95ff667aa1ff475357549da4fdd1a335207795 (172.31.14.41:31952) 负责 0 个槽位: 0-1364 5461 6827-10922 总计分配槽位: 0 / 16384 警告: 分配的总槽位数 (0) 不等于 16384 缺少 16384 个槽位分配 分片分析: 分片 4sx: 1 个主节点, 1 个从节点 ✅ 分片 58x: 1 个主节点, 1 个从节点 ✅ 分片 kss: 1 个主节点, 1 个从节点 ✅ 集群分析总结: - 检测到 3 个主节点和 3 个从节点 - 检测到 0 个故障或疑似故障节点 - 集群状态统一显示为OK: 是 - 所有槽位正确分配: 是 - 所有节点看到一致的集群大小: 是 - 所有节点看到一致的拓扑结构: 是 - 所有槽位被正确覆盖: 是 - 集群中无故障或疑似故障节点: 是 集群健康状态: 健康 ✅ 所有检查均通过，Redis集群运行正常。 analyze cluster health success `kbcli cluster describe-backup --names backup-ns-hbjkr-rediscl-epxpxf-20250528131323 --namespace ns-hbjkr ` Name: backup-ns-hbjkr-rediscl-epxpxf-20250528131323 Cluster: rediscl-epxpxf Namespace: ns-hbjkr Spec: Method: datafile Policy Name: rediscl-epxpxf-shard-backup-policy Actions: dp-backup-shard-kss-0: ActionType: Job WorkloadName: dp-backup-shard-kss-0-backup-ns-hbjkr-rediscl-epxpxf-2025052813 TargetPodName: rediscl-epxpxf-shard-kss-1 Phase: Completed Start Time: May 28,2025 13:13 UTC+0800 Completion Time: May 28,2025 13:13 UTC+0800 dp-backup-shard-4sx-0: ActionType: Job WorkloadName: dp-backup-shard-4sx-0-backup-ns-hbjkr-rediscl-epxpxf-2025052813 TargetPodName: rediscl-epxpxf-shard-4sx-1 Phase: Completed Start Time: May 28,2025 13:13 UTC+0800 Completion Time: May 28,2025 13:13 UTC+0800 dp-backup-shard-58x-0: ActionType: Job WorkloadName: dp-backup-shard-58x-0-backup-ns-hbjkr-rediscl-epxpxf-2025052813 TargetPodName: rediscl-epxpxf-shard-58x-1 Phase: Completed Start Time: May 28,2025 13:13 UTC+0800 Completion Time: May 28,2025 13:13 UTC+0800 Status: Phase: Completed Total Size: 9048 ActionSet Name: redis-cluster-br Repository: backuprepo-kbcli-test Duration: 11s Start Time: May 28,2025 13:13 UTC+0800 Completion Time: May 28,2025 13:13 UTC+0800 Path: /ns-hbjkr/rediscl-epxpxf-6ae30e0c-221c-43f1-8de6-046f4028dcbc/shard/backup-ns-hbjkr-rediscl-epxpxf-20250528131323 Warning Events: cluster connect kbcli unsupported engine type: shard-58x delete cluster rediscl-epxpxf-backup `kbcli cluster delete rediscl-epxpxf-backup --auto-approve --namespace ns-hbjkr ` Cluster rediscl-epxpxf-backup deleted pod_info:rediscl-epxpxf-backup-shard-fzg-0 3/3 Running 0 3m4s rediscl-epxpxf-backup-shard-fzg-1 3/3 Running 0 2m37s rediscl-epxpxf-backup-shard-hdn-0 3/3 Running 0 3m5s rediscl-epxpxf-backup-shard-hdn-1 3/3 Running 0 2m32s rediscl-epxpxf-backup-shard-vdl-0 3/3 Running 0 3m4s rediscl-epxpxf-backup-shard-vdl-1 3/3 Running 0 2m31s No resources found in ns-hbjkr namespace. delete cluster pod done No resources found in ns-hbjkr namespace. check cluster resource non-exist OK: pvc No resources found in ns-hbjkr namespace. delete cluster done No resources found in ns-hbjkr namespace. No resources found in ns-hbjkr namespace. No resources found in ns-hbjkr namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups backup-ns-hbjkr-rediscl-epxpxf-20250528131323 --namespace ns-hbjkr ` backup.dataprotection.kubeblocks.io/backup-ns-hbjkr-rediscl-epxpxf-20250528131323 patched `kbcli cluster delete-backup rediscl-epxpxf --name backup-ns-hbjkr-rediscl-epxpxf-20250528131323 --force --auto-approve --namespace ns-hbjkr ` Backup backup-ns-hbjkr-rediscl-epxpxf-20250528131323 deleted No opsrequests found in ns-hbjkr namespace. cluster list-logs `kbcli cluster list-logs rediscl-epxpxf --namespace ns-hbjkr ` No log files found. Error from server (NotFound): pods "rediscl-epxpxf-shard-58x-0" not found cluster logs `kbcli cluster logs rediscl-epxpxf --tail 30 --namespace ns-hbjkr ` Defaulted container "redis-cluster" out of: redis-cluster, metrics, kbagent, init-dbctl (init), init-kbagent (init), kbagent-worker (init) + local announce_bus_port=31230 ++ call_func_with_retry 3 2 send_cluster_meet 127.0.0.1 6379 172.31.10.203 30752 31230 ++ local max_retries=3 ++ local retry_interval=2 ++ local function_name=send_cluster_meet ++ shift 3 ++ local retries=0 ++ true ++ send_cluster_meet 127.0.0.1 6379 172.31.10.203 30752 31230 ++ local primary_endpoint=127.0.0.1 ++ local primary_port=6379 ++ local announce_ip=172.31.10.203 ++ local announce_port=30752 ++ local announce_bus_port=31230 ++ unset_xtrace_when_ut_mode_false ++ '[' false == false ']' ++ set +x Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. Meet the node 172.31.10.203:30752 successfully with new announce ip 172.31.10.203... + send_cluster_meet_result='check and correct other primary nodes meet command: redis-cli -h 127.0.0.1 -p 6379 -a ******** cluster meet 172.31.10.203 30752 31230 OK' + status=0 + '[' 0 -ne 0 ']' + return 0 + echo 'Meet the node 172.31.10.203 successfully with new announce ip 172.31.10.203...' + break + echo 'Node rediscl-epxpxf-shard-kss-0 is already in the cluster, skipping scale out replica...' + exit 0 Meet the node 172.31.10.203 successfully with new announce ip 172.31.10.203... Node rediscl-epxpxf-shard-kss-0 is already in the cluster, skipping scale out replica... delete cluster rediscl-epxpxf `kbcli cluster delete rediscl-epxpxf --auto-approve --namespace ns-hbjkr ` Cluster rediscl-epxpxf deleted pod_info:rediscl-epxpxf-shard-4sx-0 3/3 Running 0 21m rediscl-epxpxf-shard-4sx-1 3/3 Running 0 20m rediscl-epxpxf-shard-58x-0 3/3 Running 0 21m rediscl-epxpxf-shard-58x-1 3/3 Running 0 18m rediscl-epxpxf-shard-kss-0 3/3 Running 0 21m rediscl-epxpxf-shard-kss-1 3/3 Running 0 20m No resources found in ns-hbjkr namespace. delete cluster pod done No resources found in ns-hbjkr namespace. check cluster resource non-exist OK: pvc No resources found in ns-hbjkr namespace. delete cluster done No resources found in ns-hbjkr namespace. No resources found in ns-hbjkr namespace. No resources found in ns-hbjkr namespace. Redis Cluster Test Suite All Done! --------------------------------------Redis Cluster (Topology = redis-cluster Replicas 2) Test Result-------------------------------------- [PASSED]|[Create]|[ComponentDefinition=redis-cluster-7-1.0.0-alpha.0;ComponentVersion=redis-cluster;ServiceVersion=7.2.4;]|[Description=Create a cluster with the specified component definition redis-cluster-7-1.0.0-alpha.0 and component version redis-cluster and service version 7.2.4] [PASSED]|[Failover]|[HA=Network Duplicate;Durations=2m;ComponentName=shard-58x]|[Description=Simulates network duplicate fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to duplicate network.] [PASSED]|[Failover]|[HA=DNS Error;Durations=2m;ComponentName=shard-58x]|[Description=Simulates conditions where pods experience DNS service errors for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to DNS service errors.] [PASSED]|[Failover]|[HA=Network Loss Failover;Durations=2m;ComponentName=shard-58x]|[Description=] [PASSED]|[Failover]|[HA=Full CPU;Durations=2m;ComponentName=shard-58x]|[Description=Simulates conditions where pods experience CPU full either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high CPU load.] [PASSED]|[Failover]|[HA=Delete Pod;ComponentName=shard-58x]|[Description=Simulates conditions where pods terminating forced/graceful thereby testing deployment sanity (replica availability & uninterrupted service) and recovery workflow of the application.] [PASSED]|[Bench]|[ComponentName=shard-58x]|[Description=Bench the cluster service with shard-58x component] [PASSED]|[Failover]|[HA=Network Bandwidth Failover;Durations=2m;ComponentName=shard-58x]|[Description=] [PASSED]|[Failover]|[HA=Network Corrupt Failover;Durations=2m;ComponentName=shard-58x]|[Description=] [PASSED]|[Failover]|[HA=Network Delay;Durations=2m;ComponentName=shard-58x]|[Description=Simulates network delay fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to delay network.] [PASSED]|[Failover]|[HA=Network Partition;Durations=2m;ComponentName=shard-58x]|[Description=Simulates network partition fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to partition network.] [PASSED]|[Failover]|[HA=Kill 1;ComponentName=shard-58x]|[Description=Simulates conditions where process 1 killed either due to expected/undesired processes thereby testing the application's resilience to unavailability of some replicas due to abnormal termination signals.] [PASSED]|[VerticalScaling]|[ComponentName=shard-58x]|[Description=VerticalScaling the cluster specify component shard-58x] [PASSED]|[Failover]|[HA=OOM;Durations=2m;ComponentName=shard-58x]|[Description=Simulates conditions where pods experience OOM either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Memory load.] [PASSED]|[Failover]|[HA=DNS Random;Durations=2m;ComponentName=shard-58x]|[Description=Simulates conditions where pods experience random IP addresses being returned by the DNS service for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to the DNS service returning random IP addresses.] [PASSED]|[Failover]|[HA=Pod Failure;Durations=2m;ComponentName=shard-58x]|[Description=Simulates conditions where pods experience failure for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to failure.] [PASSED]|[Stop]|[-]|[Description=Stop the cluster] [PASSED]|[Start]|[-]|[Description=Start the cluster] [PASSED]|[Failover]|[HA=Time Offset;Durations=2m;ComponentName=shard-58x]|[Description=Simulates a time offset scenario thereby testing the application's resilience to potential slowness/unavailability of some replicas due to time offset.] [PASSED]|[HorizontalScaling Out]|[ComponentName=shard-58x]|[Description=HorizontalScaling Out the cluster specify component shard-58x] [PASSED]|[HorizontalScaling In]|[ComponentName=shard-58x]|[Description=HorizontalScaling In the cluster specify component shard-58x] [PASSED]|[VolumeExpansion]|[ComponentName=shard-58x]|[Description=VolumeExpansion the cluster specify component shard-58x] [PASSED]|[Failover]|[HA=Evicting Pod;ComponentName=shard-58x]|[Description=Simulates conditions where pods evicting either due to node drained thereby testing the application's resilience to unavailability of some replicas due to evicting.] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [PASSED]|[HorizontalScaling Out]|[ShardsName=shard]|[Description=HorizontalScaling Out the cluster] [PASSED]|[HorizontalScaling In]|[ShardsName=shard]|[Description=HorizontalScaling In the cluster] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Backup]|[Schedule=true;BackupMethod=datafile]|[Description=The cluster Schedule datafile Backup] [PASSED]|[Restore]|[Schedule=true;BackupMethod=datafile]|[Description=The cluster Schedule datafile Restore] [PASSED]|[Delete Restore Cluster]|[Schedule=true;BackupMethod=datafile]|[Description=Delete the Schedule datafile restore cluster] [PASSED]|[Backup]|[BackupMethod=datafile]|[Description=The cluster datafile Backup] [PASSED]|[Restore]|[BackupMethod=datafile]|[Description=The cluster datafile Restore] [PASSED]|[Delete Restore Cluster]|[BackupMethod=datafile]|[Description=Delete the datafile restore cluster] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]