source commons files source engines files source kubeblocks files source kubedb files CLUSTER_NAME: `kubectl get namespace | grep ns-cwzhr ` `kubectl create namespace ns-cwzhr` namespace/ns-cwzhr created create namespace ns-cwzhr done download kbcli `gh release list --repo apecloud/kbcli --limit 100 | (grep "1.0" || true)` `curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v1.0.1` Your system is linux_amd64 Installing kbcli ... Downloading ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 33.6M 100 33.6M 0 0 71.0M 0 --:--:-- --:--:-- --:--:-- 71.0M kbcli installed successfully. Kubernetes: v1.32.6 KubeBlocks: 1.0.1 kbcli: 1.0.1 Make sure your docker service is running and begin your journey with kbcli: kbcli playground init For more information on how to get started, please visit: https://kubeblocks.io download kbcli v1.0.1 done Kubernetes: v1.32.6 KubeBlocks: 1.0.1 kbcli: 1.0.1 Kubernetes Env: v1.32.6 check snapshot controller check snapshot controller done POD_RESOURCES: aks kb-default-sc found aks default-vsc found found default storage class: default KubeBlocks version is:1.0.1 skip upgrade KubeBlocks current KubeBlocks version: 1.0.1 Error: no repositories to show helm repo add chaos-mesh https://charts.chaos-mesh.org "chaos-mesh" has been added to your repositories add helm chart repo chaos-mesh success chaos mesh already installed check component definition set component name:redis set component version set component version:redis set service versions:8.2.1,8.0.3,8.0.1,7.4.5,7.4.2,7.2.10,7.2.7,7.2.4,7.0.6,6.2.17,6.2.14,5.0.12 set service versions sorted:5.0.12,6.2.14,6.2.17,7.0.6,7.2.4,7.2.7,7.2.10,7.4.2,7.4.5,8.0.1,8.0.3,8.2.1 set redis component definition set redis component definition redis-cluster-6-1.0.1 REPORT_COUNT 0:0 set replicas first:2,6.2.14|2,6.2.17|2,7.0.6|2,7.2.4|2,7.2.7|2,7.2.10|2,7.4.2|2,7.4.5|2,8.0.1|2,8.0.3|2,8.2.1 set replicas third:2,6.2.14 set replicas fourth:2,6.2.14 set minimum cmpv service version set minimum cmpv service version replicas:2,6.2.14 REPORT_COUNT:1 CLUSTER_TOPOLOGY:replication topology replication found in cluster definition redis set redis component definition set redis component definition redis-5-1.0.1 LIMIT_CPU:0.1 LIMIT_MEMORY:0.5 storage size: 1 CLUSTER_NAME:redis-zgrujr No resources found in ns-cwzhr namespace. pod_info: termination_policy:WipeOut create 2 replica WipeOut redis cluster check component definition set component definition by component version check cmpd by labels set component definition1: redis-6-1.0.1 by component version:redis apiVersion: apps.kubeblocks.io/v1 kind: Cluster metadata: name: redis-zgrujr namespace: ns-cwzhr spec: clusterDef: redis topology: replication terminationPolicy: WipeOut componentSpecs: - name: redis serviceVersion: 6.2.14 replicas: 2 disableExporter: false resources: limits: cpu: 100m memory: 0.5Gi requests: cpu: 100m memory: 0.5Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi - name: redis-sentinel serviceVersion: 6.2.14 replicas: 3 resources: limits: cpu: 100m memory: 0.5Gi requests: cpu: 100m memory: 0.5Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi `kubectl apply -f test_create_redis-zgrujr.yaml` cluster.apps.kubeblocks.io/redis-zgrujr created apply test_create_redis-zgrujr.yaml Success `rm -rf test_create_redis-zgrujr.yaml` check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Creating Sep 11,2025 20:24 UTC+0800 clusterdefinition.kubeblocks.io/name=redis cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:26 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:27 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:24 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check pod redis-zgrujr-redis-0 container_name redis exist password 1A26G0F9bC check pod redis-zgrujr-redis-0 container_name kbagent exist password 1A26G0F9bC check pod redis-zgrujr-redis-0 container_name config-manager exist password 1A26G0F9bC No container logs contain secret password. describe cluster `kbcli cluster describe redis-zgrujr --namespace ns-cwzhr ` Name: redis-zgrujr Created Time: Sep 11,2025 20:24 UTC+0800 NAMESPACE CLUSTER-DEFINITION TOPOLOGY STATUS TERMINATION-POLICY ns-cwzhr redis replication Running WipeOut Endpoints: COMPONENT INTERNAL EXTERNAL redis redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local:6379 redis-sentinel redis-zgrujr-redis-sentinel-redis-sentinel.ns-cwzhr.svc.cluster.local:26379 Topology: COMPONENT SERVICE-VERSION INSTANCE ROLE STATUS AZ NODE CREATED-TIME redis 6.2.14 redis-zgrujr-redis-0 primary Running 0 aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:26 UTC+0800 redis 6.2.14 redis-zgrujr-redis-1 secondary Running 0 aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:27 UTC+0800 redis-sentinel 6.2.14 redis-zgrujr-redis-sentinel-0 Running 0 aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:24 UTC+0800 redis-sentinel 6.2.14 redis-zgrujr-redis-sentinel-1 Running 0 aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:24 UTC+0800 redis-sentinel 6.2.14 redis-zgrujr-redis-sentinel-2 Running 0 aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:24 UTC+0800 Resources Allocation: COMPONENT INSTANCE-TEMPLATE CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS redis 100m / 100m 512Mi / 512Mi data:1Gi default redis-sentinel 100m / 100m 512Mi / 512Mi data:1Gi default Images: COMPONENT COMPONENT-DEFINITION IMAGE redis redis-6-1.0.1 docker.io/apecloud/redis-stack-server:6.2.6-v10 docker.io/apecloud/redis-stack-server:6.2.6-v18 redis-sentinel redis-sentinel-6-1.0.1 docker.io/apecloud/redis-stack-server:6.2.6-v10 Data Protection: BACKUP-REPO AUTO-BACKUP BACKUP-SCHEDULE BACKUP-METHOD BACKUP-RETENTION RECOVERABLE-TIME Show cluster events: kbcli cluster list-events -n ns-cwzhr redis-zgrujr `kbcli cluster label redis-zgrujr app.kubernetes.io/instance- --namespace ns-cwzhr ` label "app.kubernetes.io/instance" not found. `kbcli cluster label redis-zgrujr app.kubernetes.io/instance=redis-zgrujr --namespace ns-cwzhr ` `kbcli cluster label redis-zgrujr --list --namespace ns-cwzhr ` NAME NAMESPACE LABELS redis-zgrujr ns-cwzhr app.kubernetes.io/instance=redis-zgrujr clusterdefinition.kubeblocks.io/name=redis label cluster app.kubernetes.io/instance=redis-zgrujr Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=redis-zgrujr --namespace ns-cwzhr ` `kbcli cluster label redis-zgrujr --list --namespace ns-cwzhr ` NAME NAMESPACE LABELS redis-zgrujr ns-cwzhr app.kubernetes.io/instance=redis-zgrujr case.name=kbcli.test1 clusterdefinition.kubeblocks.io/name=redis label cluster case.name=kbcli.test1 Success `kbcli cluster label redis-zgrujr case.name=kbcli.test2 --overwrite --namespace ns-cwzhr ` `kbcli cluster label redis-zgrujr --list --namespace ns-cwzhr ` NAME NAMESPACE LABELS redis-zgrujr ns-cwzhr app.kubernetes.io/instance=redis-zgrujr case.name=kbcli.test2 clusterdefinition.kubeblocks.io/name=redis label cluster case.name=kbcli.test2 Success `kbcli cluster label redis-zgrujr case.name- --namespace ns-cwzhr ` `kbcli cluster label redis-zgrujr --list --namespace ns-cwzhr ` NAME NAMESPACE LABELS redis-zgrujr ns-cwzhr app.kubernetes.io/instance=redis-zgrujr clusterdefinition.kubeblocks.io/name=redis delete cluster label case.name Success cluster connect `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. # Server redis_version:6.2.14 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:33f32d76680500a7 redis_mode:standalone os:Linux 5.15.0-1092-azure x86_64 arch_bits:64 monotonic_clock:POSIX clock_gettime multiplexing_api:epoll atomicvar_api:c11-builtin gcc_version:9.4.0 process_id:1 process_supervised:no run_id:fa0ce7d313865537a419dfd09942f82e690aae2c tcp_port:6379 server_time_usec:1757593725115685 uptime_in_seconds:97 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:12763261 executable:/redis-server config_file:/etc/redis/redis.conf io_threads_active:0 `echo " echo \"CONFIG GET maxclients\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. maxclients 10000 connect cluster Success set max_connections to 10000 insert batch data by db client Error from server (NotFound): pods "test-db-client-executionloop-redis-zgrujr" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-redis-zgrujr --namespace ns-cwzhr ` Error from server (NotFound): pods "test-db-client-executionloop-redis-zgrujr" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-executionloop-redis-zgrujr" not found `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 apiVersion: v1 kind: Pod metadata: name: test-db-client-executionloop-redis-zgrujr namespace: ns-cwzhr spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local" - "--user" - "default" - "--password" - "1A26G0F9bC" - "--port" - "6379" - "--dbtype" - "redis" - "--test" - "executionloop" - "--duration" - "60" - "--interval" - "1" restartPolicy: Never `kubectl apply -f test-db-client-executionloop-redis-zgrujr.yaml` pod/test-db-client-executionloop-redis-zgrujr created apply test-db-client-executionloop-redis-zgrujr.yaml Success `rm -rf test-db-client-executionloop-redis-zgrujr.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 1/1 Running 0 5s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 1/1 Running 0 10s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 1/1 Running 0 15s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 1/1 Running 0 20s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 1/1 Running 0 25s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 1/1 Running 0 31s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 1/1 Running 0 36s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 1/1 Running 0 41s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 1/1 Running 0 46s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 1/1 Running 0 52s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 1/1 Running 0 57s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 1/1 Running 0 62s check pod test-db-client-executionloop-redis-zgrujr status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-redis-zgrujr 0/1 Completed 0 68s check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:26 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:27 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:24 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done [2025-09-11 12:28:55.032] Connection error occurred! [ 2s ] executions total: 801 successful: 800 failed: 1 disconnect: 1 [2025-09-11 12:28:55.032] Connection error occurred! [2025-09-11 12:28:56.035] Connection successfully recovered! The connection was restored in 1003 milliseconds. [ 3s ] executions total: 802 successful: 801 failed: 1 disconnect: 1 [ 4s ] executions total: 1550 successful: 1549 failed: 1 disconnect: 1 [2025-09-11 12:29:00.129] Connection error occurred! [ 5s ] executions total: 1602 successful: 1600 failed: 2 disconnect: 2 [2025-09-11 12:29:00.129] Connection error occurred! [2025-09-11 12:29:01.131] Connection successfully recovered! The connection was restored in 1002 milliseconds. [ 6s ] executions total: 1603 successful: 1601 failed: 2 disconnect: 2 [ 7s ] executions total: 2367 successful: 2365 failed: 2 disconnect: 2 [2025-09-11 12:29:05.220] Connection error occurred! [ 8s ] executions total: 2403 successful: 2400 failed: 3 disconnect: 3 [2025-09-11 12:29:05.220] Connection error occurred! [2025-09-11 12:29:06.222] Connection successfully recovered! The connection was restored in 1002 milliseconds. [ 9s ] executions total: 2404 successful: 2401 failed: 3 disconnect: 3 [ 10s ] executions total: 3176 successful: 3173 failed: 3 disconnect: 3 [2025-09-11 12:29:10.231] Connection error occurred! [ 11s ] executions total: 3204 successful: 3200 failed: 4 disconnect: 4 [2025-09-11 12:29:10.231] Connection error occurred! [2025-09-11 12:29:11.233] Connection successfully recovered! The connection was restored in 1002 milliseconds. [ 12s ] executions total: 3205 successful: 3201 failed: 4 disconnect: 4 [ 13s ] executions total: 3995 successful: 3991 failed: 4 disconnect: 4 [2025-09-11 12:29:15.235] Connection error occurred! [ 14s ] executions total: 4005 successful: 4000 failed: 5 disconnect: 5 [2025-09-11 12:29:15.235] Connection error occurred! [2025-09-11 12:29:16.237] Connection successfully recovered! The connection was restored in 1002 milliseconds. [ 15s ] executions total: 4006 successful: 4001 failed: 5 disconnect: 5 [2025-09-11 12:29:20.222] Connection error occurred! [ 16s ] executions total: 4806 successful: 4800 failed: 6 disconnect: 6 [2025-09-11 12:29:20.222] Connection error occurred! [2025-09-11 12:29:21.232] Connection successfully recovered! The connection was restored in 1010 milliseconds. [ 17s ] executions total: 4807 successful: 4801 failed: 6 disconnect: 6 [ 18s ] executions total: 5606 successful: 5600 failed: 6 disconnect: 6 [2025-09-11 12:29:25.232] Connection error occurred! [ 19s ] executions total: 5607 successful: 5600 failed: 7 disconnect: 7 [2025-09-11 12:29:25.232] Connection error occurred! [2025-09-11 12:29:26.234] Connection successfully recovered! The connection was restored in 1002 milliseconds. [ 20s ] executions total: 5608 successful: 5601 failed: 7 disconnect: 7 [ 21s ] executions total: 6397 successful: 6390 failed: 7 disconnect: 7 [2025-09-11 12:29:30.316] Connection error occurred! [ 22s ] executions total: 6408 successful: 6400 failed: 8 disconnect: 8 [2025-09-11 12:29:30.316] Connection error occurred! [2025-09-11 12:29:31.318] Connection successfully recovered! The connection was restored in 1002 milliseconds. [ 23s ] executions total: 6409 successful: 6401 failed: 8 disconnect: 8 [ 24s ] executions total: 7137 successful: 7129 failed: 8 disconnect: 8 [2025-09-11 12:29:35.334] Connection error occurred! [ 25s ] executions total: 7209 successful: 7200 failed: 9 disconnect: 9 [2025-09-11 12:29:35.334] Connection error occurred! [2025-09-11 12:29:36.337] Connection successfully recovered! The connection was restored in 1003 milliseconds. [ 26s ] executions total: 7210 successful: 7201 failed: 9 disconnect: 9 [2025-09-11 12:29:40.331] Connection error occurred! [ 27s ] executions total: 8010 successful: 8000 failed: 10 disconnect: 10 [2025-09-11 12:29:40.331] Connection error occurred! [2025-09-11 12:29:41.332] Connection successfully recovered! The connection was restored in 1001 milliseconds. [ 28s ] executions total: 8011 successful: 8001 failed: 10 disconnect: 10 [2025-09-11 12:29:45.316] Connection error occurred! [ 29s ] executions total: 8811 successful: 8800 failed: 11 disconnect: 11 [2025-09-11 12:29:45.316] Connection error occurred! [2025-09-11 12:29:46.318] Connection successfully recovered! The connection was restored in 1002 milliseconds. [ 30s ] executions total: 8812 successful: 8801 failed: 11 disconnect: 11 [ 31s ] executions total: 9547 successful: 9536 failed: 11 disconnect: 11 [2025-09-11 12:29:50.417] Connection error occurred! [ 32s ] executions total: 9612 successful: 9600 failed: 12 disconnect: 12 [2025-09-11 12:29:50.417] Connection error occurred! [2025-09-11 12:29:51.437] Connection successfully recovered! The connection was restored in 1020 milliseconds. [ 60s ] executions total: 9613 successful: 9601 failed: 12 disconnect: 12 Test Result: Total Executions: 9613 Successful Executions: 9601 Failed Executions: 12 Disconnection Counts: 12 Connection Information: Database Type: redis Host: redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local Port: 6379 Database: Table: User: default Org: Access Mode: mysql Test Type: executionloop Query: Duration: 60 seconds Interval: 1 seconds Key: DB_CLIENT_BATCH_DATA_COUNT: 9601 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-redis-zgrujr --namespace ns-cwzhr ` pod/test-db-client-executionloop-redis-zgrujr patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-executionloop-redis-zgrujr" force deleted `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo 'echo "set mykey \"nlbzw\"" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. OK add consistent data nlbzw Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo 'echo "set mykey \"nlbzw\"" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. READONLY You can't write against a read only replica. check add consistent data readonly Success LB_TYPE is set to: intranet cluster expose check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster expose redis-zgrujr --auto-approve --force=true --type intranet --enable true --components redis --role-selector primary --namespace ns-cwzhr ` OpsRequest redis-zgrujr-expose-566lc created successfully, you can view the progress: kbcli cluster describe-ops redis-zgrujr-expose-566lc -n ns-cwzhr check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:26 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:27 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:24 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 20:30 UTC+0800 check ops status done ops_status:redis-zgrujr-expose-566lc ns-cwzhr Expose redis-zgrujr redis Succeed 1/1 Sep 11,2025 20:30 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations redis-zgrujr-expose-566lc --namespace ns-cwzhr ` opsrequest.operations.kubeblocks.io/redis-zgrujr-expose-566lc patched `kbcli cluster delete-ops --name redis-zgrujr-expose-566lc --force --auto-approve --namespace ns-cwzhr ` OpsRequest redis-zgrujr-expose-566lc deleted `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success test failover connectionstress check cluster status before cluster-failover-connectionstress check cluster status done cluster_status:Running Error from server (NotFound): pods "test-db-client-connectionstress-redis-zgrujr" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstress-redis-zgrujr --namespace ns-cwzhr ` Error from server (NotFound): pods "test-db-client-connectionstress-redis-zgrujr" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-connectionstress-redis-zgrujr" not found `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 apiVersion: v1 kind: Pod metadata: name: test-db-client-connectionstress-redis-zgrujr namespace: ns-cwzhr spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local" - "--user" - "default" - "--password" - "1A26G0F9bC" - "--port" - "6379" - "--database" - "db0" - "--dbtype" - "redis" - "--test" - "connectionstress" - "--connections" - "10000" - "--duration" - "60" restartPolicy: Never `kubectl apply -f test-db-client-connectionstress-redis-zgrujr.yaml` pod/test-db-client-connectionstress-redis-zgrujr created apply test-db-client-connectionstress-redis-zgrujr.yaml Success `rm -rf test-db-client-connectionstress-redis-zgrujr.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-redis-zgrujr 1/1 Running 0 6s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-redis-zgrujr 1/1 Running 0 10s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-redis-zgrujr 1/1 Running 0 15s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-redis-zgrujr 1/1 Running 0 20s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-redis-zgrujr 1/1 Running 0 26s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-redis-zgrujr 1/1 Running 0 31s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-redis-zgrujr 1/1 Running 0 36s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-redis-zgrujr 1/1 Running 0 42s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-redis-zgrujr 1/1 Running 0 47s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-redis-zgrujr 1/1 Running 0 52s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-redis-zgrujr 1/1 Running 0 57s check pod test-db-client-connectionstress-redis-zgrujr status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-redis-zgrujr 0/1 Completed 0 63s check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:26 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:27 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:24 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done --host redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local --user default --password 1A26G0F9bC --port 6379 --database db0 --dbtype redis --test connectionstress --connections 10000 --duration 60 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider] Test Result: Connection stress test results: Duration: 60 seconds Successful connections: 9898 Failed connections: 0 Connection Information: Database Type: redis Host: redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local Port: 6379 Database: db0 Table: User: default Org: Access Mode: mysql Test Type: connectionstress Connection Count: 10000 Duration: 60 seconds `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstress-redis-zgrujr --namespace ns-cwzhr ` pod/test-db-client-connectionstress-redis-zgrujr patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-connectionstress-redis-zgrujr" force deleted check failover pod name failover pod name:redis-zgrujr-redis-0 failover connectionstress Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success 9 get redis-sentinel replicas test failover podkill check cluster status before cluster-failover-podkill check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podkill-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podkill-redis-zgrujr" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podkill-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podkill-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-0 mode: all action: pod-kill `kubectl apply -f test-chaos-mesh-podkill-redis-zgrujr.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podkill-redis-zgrujr created apply test-chaos-mesh-podkill-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-podkill-redis-zgrujr.yaml` check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Updating Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:33 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:27 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:24 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podkill-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podkill-redis-zgrujr" force deleted Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podkill-redis-zgrujr" not found check failover pod name failover pod name:redis-zgrujr-redis-1 failover podkill Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 apiVersion: v1 kind: Pod metadata: name: benchtest-redis-zgrujr namespace: ns-cwzhr spec: containers: - name: test-benchmark imagePullPolicy: IfNotPresent image: docker.io/apecloud/redis-benchmark:latest args: - "-h" - "redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local" - "-p" - "6379" - "-a" - "1A26G0F9bC" - "-n" - "1000" - "-c" - "4" - "-q" restartPolicy: Never `kubectl apply -f benchtest-redis-zgrujr.yaml` pod/benchtest-redis-zgrujr created apply benchtest-redis-zgrujr.yaml Success check pod status pod_status:NAME READY STATUS RESTARTS AGE benchtest-redis-zgrujr 0/1 ContainerCreating 0 1s pod_status:NAME READY STATUS RESTARTS AGE benchtest-redis-zgrujr 0/1 ContainerCreating 0 5s pod_status:NAME READY STATUS RESTARTS AGE benchtest-redis-zgrujr 1/1 Running 0 10s check pod benchtest-redis-zgrujr status done pod_status:NAME READY STATUS RESTARTS AGE benchtest-redis-zgrujr 0/1 Completed 0 16s `rm -rf benchtest-redis-zgrujr.yaml` `kubectl logs benchtest-redis-zgrujr --tail 30 --namespace ns-cwzhr ` PING_INLINE: rps=0.0 (overall: -nan) avg_msec=-nan (overall: -nan) PING_INLINE: 9009.01 requests per second, p50=0.111 msec PING_MBULK: 10869.57 requests per second, p50=0.119 msec SET: rps=0.0 (overall: -nan) avg_msec=-nan (overall: -nan) SET: rps=3362.6 (overall: 3362.6) avg_msec=1.176 (overall: 1.176) SET: 3891.05 requests per second, p50=0.127 msec GET: 5882.35 requests per second, p50=0.119 msec INCR: rps=920.3 (overall: 10043.5) avg_msec=0.143 (overall: 0.143) INCR: 3703.70 requests per second, p50=0.135 msec LPUSH: rps=0.0 (overall: -nan) avg_msec=-nan (overall: -nan) LPUSH: rps=3789.7 (overall: 3789.7) avg_msec=0.767 (overall: 0.767) LPUSH: 3610.11 requests per second, p50=0.127 msec RPUSH: rps=3792.0 (overall: 4812.2) avg_msec=0.760 (overall: 0.760) RPUSH: 3663.00 requests per second, p50=0.127 msec LPOP: rps=2753.0 (overall: 4487.0) avg_msec=0.519 (overall: 0.519) LPOP: 5319.15 requests per second, p50=0.127 msec RPOP: rps=3167.3 (overall: 4076.9) avg_msec=0.894 (overall: 0.894) RPOP: 3649.64 requests per second, p50=0.127 msec SADD: rps=3382.5 (overall: 5895.8) avg_msec=0.373 (overall: 0.373) SADD: 5882.35 requests per second, p50=0.119 msec HSET: 5524.86 requests per second, p50=0.135 msec SPOP: rps=0.0 (overall: -nan) avg_msec=-nan (overall: -nan) SPOP: rps=2677.3 (overall: 2677.3) avg_msec=0.957 (overall: 0.957) SPOP: 2732.24 requests per second, p50=0.111 msec ZADD: 11111.11 requests per second, p50=0.111 msec ZPOPMIN: rps=0.0 (overall: -nan) avg_msec=-nan (overall: -nan) ZPOPMIN: 5847.95 requests per second, p50=0.111 msec LPUSH (needed to benchmark LRANGE): rps=458.2 (overall: 2090.9) avg_msec=0.135 (overall: 0.135) LPUSH (needed to benchmark LRANGE): 3759.40 requests per second, p50=0.119 msec LRANGE_100 (first 100 elements): rps=637.5 (overall: 10666.7) avg_msec=0.153 (overall: 0.153) LRANGE_100 (first 100 elements): rps=2972.1 (overall: 3406.0) avg_msec=0.921 (overall: 0.785) LRANGE_100 (first 100 elements): 3676.47 requests per second, p50=0.135 msec LRANGE_300 (first 300 elements): rps=3039.8 (overall: 3406.2) avg_msec=0.930 (overall: 0.930) LRANGE_300 (first 300 elements): 3412.97 requests per second, p50=0.175 msec LRANGE_500 (first 500 elements): rps=725.1 (overall: 1174.2) avg_msec=1.408 (overall: 1.408) LRANGE_500 (first 500 elements): rps=2220.0 (overall: 1819.8) avg_msec=2.013 (overall: 1.864) LRANGE_500 (first 500 elements): 1776.20 requests per second, p50=0.191 msec LRANGE_600 (first 600 elements): rps=286.9 (overall: 1014.1) avg_msec=3.756 (overall: 3.756) LRANGE_600 (first 600 elements): rps=1956.2 (overall: 1748.4) avg_msec=1.599 (overall: 1.874) LRANGE_600 (first 600 elements): rps=1624.0 (overall: 1694.1) avg_msec=2.569 (overall: 2.166) LRANGE_600 (first 600 elements): 1733.10 requests per second, p50=0.287 msec MSET (10 keys): rps=1788.8 (overall: 2050.2) avg_msec=1.536 (overall: 1.536) MSET (10 keys): 2141.33 requests per second, p50=0.151 msec XADD: rps=0.0 (overall: -nan) avg_msec=-nan (overall: -nan) XADD: 5586.59 requests per second, p50=0.119 msec `kubectl delete pod benchtest-redis-zgrujr --force --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "benchtest-redis-zgrujr" force deleted LB_TYPE is set to: intranet No resources found in ns-cwzhr namespace. `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 apiVersion: v1 kind: Pod metadata: name: benchtest-redis-zgrujr namespace: ns-cwzhr spec: containers: - name: test-benchmark imagePullPolicy: IfNotPresent image: docker.io/apecloud/redis-benchmark:latest args: - "-h" - "10.224.0.6" - "-p" - "6379" - "-a" - "1A26G0F9bC" - "-n" - "1000" - "-c" - "4" - "-q" restartPolicy: Never `kubectl apply -f benchtest-redis-zgrujr.yaml` pod/benchtest-redis-zgrujr created apply benchtest-redis-zgrujr.yaml Success check pod status pod_status:NAME READY STATUS RESTARTS AGE benchtest-redis-zgrujr 0/1 ContainerCreating 0 0s pod_status:NAME READY STATUS RESTARTS AGE benchtest-redis-zgrujr 1/1 Running 0 4s check pod benchtest-redis-zgrujr status done pod_status:NAME READY STATUS RESTARTS AGE benchtest-redis-zgrujr 0/1 Completed 0 9s `rm -rf benchtest-redis-zgrujr.yaml` `kubectl logs benchtest-redis-zgrujr --tail 30 --namespace ns-cwzhr ` PING_INLINE: rps=0.0 (overall: -nan) avg_msec=-nan (overall: -nan) PING_INLINE: 10204.08 requests per second, p50=0.111 msec PING_MBULK: 9009.01 requests per second, p50=0.111 msec SET: rps=111.6 (overall: 700.0) avg_msec=0.149 (overall: 0.149) SET: 3460.21 requests per second, p50=0.135 msec GET: rps=4.0 (overall: 1000.0) avg_msec=0.132 (overall: 0.132) GET: 5076.14 requests per second, p50=0.111 msec INCR: rps=944.4 (overall: 4327.3) avg_msec=0.126 (overall: 0.126) INCR: 3322.26 requests per second, p50=0.127 msec LPUSH: rps=99.2 (overall: 4166.7) avg_msec=0.163 (overall: 0.163) LPUSH: rps=2470.1 (overall: 2509.7) avg_msec=1.292 (overall: 1.249) LPUSH: 3289.47 requests per second, p50=0.135 msec RPUSH: rps=3179.3 (overall: 3911.8) avg_msec=0.999 (overall: 0.999) RPUSH: 3401.36 requests per second, p50=0.135 msec LPOP: rps=1722.2 (overall: 2695.7) avg_msec=0.928 (overall: 0.928) LPOP: 3289.47 requests per second, p50=0.143 msec RPOP: rps=1896.0 (overall: 4471.7) avg_msec=0.830 (overall: 0.830) RPOP: 3389.83 requests per second, p50=0.143 msec SADD: rps=920.3 (overall: 3786.9) avg_msec=0.168 (overall: 0.168) SADD: 5128.21 requests per second, p50=0.167 msec HSET: rps=1482.1 (overall: 3179.5) avg_msec=1.058 (overall: 1.058) HSET: rps=1804.0 (overall: 2242.5) avg_msec=1.838 (overall: 1.485) HSET: 2481.39 requests per second, p50=0.135 msec SPOP: 5025.13 requests per second, p50=0.119 msec ZADD: rps=1170.6 (overall: 18437.5) avg_msec=0.124 (overall: 0.124) ZADD: 5128.21 requests per second, p50=0.119 msec ZPOPMIN: rps=2119.5 (overall: 7388.9) avg_msec=0.123 (overall: 0.123) ZPOPMIN: 8695.65 requests per second, p50=0.127 msec LPUSH (needed to benchmark LRANGE): rps=2685.3 (overall: 3256.0) avg_msec=1.097 (overall: 1.097) LPUSH (needed to benchmark LRANGE): 2610.97 requests per second, p50=0.143 msec LRANGE_100 (first 100 elements): rps=1223.1 (overall: 4093.3) avg_msec=0.142 (overall: 0.142) LRANGE_100 (first 100 elements): 3333.33 requests per second, p50=0.143 msec LRANGE_300 (first 300 elements): rps=729.1 (overall: 7320.0) avg_msec=0.193 (overall: 0.193) LRANGE_300 (first 300 elements): rps=2131.5 (overall: 2601.4) avg_msec=1.427 (overall: 1.113) LRANGE_300 (first 300 elements): 2512.56 requests per second, p50=0.183 msec LRANGE_500 (first 500 elements): rps=1589.6 (overall: 3117.2) avg_msec=1.035 (overall: 1.035) LRANGE_500 (first 500 elements): rps=1396.8 (overall: 1976.3) avg_msec=2.090 (overall: 1.529) LRANGE_500 (first 500 elements): 1996.01 requests per second, p50=0.247 msec LRANGE_600 (first 600 elements): rps=1099.6 (overall: 2123.1) avg_msec=1.506 (overall: 1.506) LRANGE_600 (first 600 elements): rps=1028.0 (overall: 1402.6) avg_msec=2.817 (overall: 2.138) LRANGE_600 (first 600 elements): 1631.32 requests per second, p50=0.271 msec MSET (10 keys): rps=345.2 (overall: 4833.3) avg_msec=0.165 (overall: 0.165) MSET (10 keys): rps=2175.3 (overall: 2353.2) avg_msec=1.389 (overall: 1.221) MSET (10 keys): 2570.69 requests per second, p50=0.151 msec XADD: rps=1968.1 (overall: 3771.0) avg_msec=0.848 (overall: 0.848) XADD: 3300.33 requests per second, p50=0.143 msec `kubectl delete pod benchtest-redis-zgrujr --force --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "benchtest-redis-zgrujr" force deleted test failover networkcorruptover check cluster status before cluster-failover-networkcorruptover check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-redis-zgrujr" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorruptover-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-1 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorruptover-redis-zgrujr.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-redis-zgrujr created apply test-chaos-mesh-networkcorruptover-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-networkcorruptover-redis-zgrujr.yaml` networkcorruptover chaos test waiting 120 seconds check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:33 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:27 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:24 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-redis-zgrujr" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-redis-zgrujr" not found check failover pod name failover pod name:redis-zgrujr-redis-0 failover networkcorruptover Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success cluster does not need to check monitor currently check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:33 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:27 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:24 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done test failover kill1 check cluster status before cluster-failover-kill1 check cluster status done cluster_status:Running `kill 1` Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file exec return message: check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Updating Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:33 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:27 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:24 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done check failover pod name failover pod name:redis-zgrujr-redis-0 checking failover... `kill 1` Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file exec return message: check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis NotReady primary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:33 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:27 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:24 UTC+0800 pod_status:NotReady pod_status:CrashLoopBackOff pod_status:CrashLoopBackOff check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done check failover pod name:redis-zgrujr-redis-1 failover kill1 Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success test failover dnsrandom check cluster status before cluster-failover-dnsrandom cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-redis-zgrujr" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnsrandom-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-1 mode: all action: random duration: 2m `kubectl apply -f test-chaos-mesh-dnsrandom-redis-zgrujr.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnsrandom-redis-zgrujr created apply test-chaos-mesh-dnsrandom-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-dnsrandom-redis-zgrujr.yaml` dnsrandom chaos test waiting 120 seconds check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:33 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:27 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:24 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-redis-zgrujr" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-redis-zgrujr" not found check failover pod name failover pod name:redis-zgrujr-redis-1 failover dnsrandom Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success `kubectl get pvc -l app.kubernetes.io/instance=redis-zgrujr,apps.kubeblocks.io/component-name=redis,apps.kubeblocks.io/vct-name=data --namespace ns-cwzhr ` cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in redis-zgrujr namespace. `kbcli cluster volume-expand redis-zgrujr --auto-approve --force=true --components redis --volume-claim-templates data --storage 6Gi --namespace ns-cwzhr ` OpsRequest redis-zgrujr-volumeexpansion-hlhgd created successfully, you can view the progress: kbcli cluster describe-ops redis-zgrujr-volumeexpansion-hlhgd -n ns-cwzhr check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-volumeexpansion-hlhgd ns-cwzhr VolumeExpansion redis-zgrujr redis Running 0/2 Sep 11,2025 20:43 UTC+0800 check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Updating Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:33 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:27 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:24 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:24 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done No resources found in redis-zgrujr namespace. check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-volumeexpansion-hlhgd ns-cwzhr VolumeExpansion redis-zgrujr redis Succeed 2/2 Sep 11,2025 20:43 UTC+0800 check ops status done ops_status:redis-zgrujr-volumeexpansion-hlhgd ns-cwzhr VolumeExpansion redis-zgrujr redis Succeed 2/2 Sep 11,2025 20:43 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations redis-zgrujr-volumeexpansion-hlhgd --namespace ns-cwzhr ` opsrequest.operations.kubeblocks.io/redis-zgrujr-volumeexpansion-hlhgd patched `kbcli cluster delete-ops --name redis-zgrujr-volumeexpansion-hlhgd --force --auto-approve --namespace ns-cwzhr ` OpsRequest redis-zgrujr-volumeexpansion-hlhgd deleted `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success cluster stop check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster stop redis-zgrujr --auto-approve --force=true --namespace ns-cwzhr ` OpsRequest redis-zgrujr-stop-j4wc5 created successfully, you can view the progress: kbcli cluster describe-ops redis-zgrujr-stop-j4wc5 -n ns-cwzhr check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-stop-j4wc5 ns-cwzhr Stop redis-zgrujr redis,redis-sentinel Running 0/5 Sep 11,2025 20:48 UTC+0800 check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Stopped Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Stopped check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME check pod status done check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-stop-j4wc5 ns-cwzhr Stop redis-zgrujr redis,redis-sentinel Succeed 5/5 Sep 11,2025 20:48 UTC+0800 check ops status done ops_status:redis-zgrujr-stop-j4wc5 ns-cwzhr Stop redis-zgrujr redis,redis-sentinel Succeed 5/5 Sep 11,2025 20:48 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations redis-zgrujr-stop-j4wc5 --namespace ns-cwzhr ` opsrequest.operations.kubeblocks.io/redis-zgrujr-stop-j4wc5 patched `kbcli cluster delete-ops --name redis-zgrujr-stop-j4wc5 --force --auto-approve --namespace ns-cwzhr ` OpsRequest redis-zgrujr-stop-j4wc5 deleted cluster start check cluster status before ops check cluster status done cluster_status:Stopped `kbcli cluster start redis-zgrujr --force=true --namespace ns-cwzhr ` OpsRequest redis-zgrujr-start-c5mhp created successfully, you can view the progress: kbcli cluster describe-ops redis-zgrujr-start-c5mhp -n ns-cwzhr check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-start-c5mhp ns-cwzhr Start redis-zgrujr redis,redis-sentinel Running 0/5 Sep 11,2025 20:48 UTC+0800 check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Updating Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:49 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:50 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-start-c5mhp ns-cwzhr Start redis-zgrujr redis,redis-sentinel Succeed 5/5 Sep 11,2025 20:48 UTC+0800 check ops status done ops_status:redis-zgrujr-start-c5mhp ns-cwzhr Start redis-zgrujr redis,redis-sentinel Succeed 5/5 Sep 11,2025 20:48 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations redis-zgrujr-start-c5mhp --namespace ns-cwzhr ` opsrequest.operations.kubeblocks.io/redis-zgrujr-start-c5mhp patched `kbcli cluster delete-ops --name redis-zgrujr-start-c5mhp --force --auto-approve --namespace ns-cwzhr ` OpsRequest redis-zgrujr-start-c5mhp deleted `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success test failover podfailure check cluster status before cluster-failover-podfailure check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-redis-zgrujr" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podfailure-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-1 mode: all action: pod-failure duration: 2m `kubectl apply -f test-chaos-mesh-podfailure-redis-zgrujr.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podfailure-redis-zgrujr created apply test-chaos-mesh-podfailure-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-podfailure-redis-zgrujr.yaml` podfailure chaos test waiting 120 seconds check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Updating Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:49 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:50 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-redis-zgrujr" force deleted Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-redis-zgrujr" not found check failover pod name failover pod name:redis-zgrujr-redis-0 failover podfailure Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success test failover oom check cluster status before cluster-failover-oom check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-redis-zgrujr" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-0 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-redis-zgrujr.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-redis-zgrujr created apply test-chaos-mesh-oom-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-oom-redis-zgrujr.yaml` check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Updating Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:49 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:50 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-redis-zgrujr" force deleted stresschaos.chaos-mesh.org/test-chaos-mesh-oom-redis-zgrujr patched check failover pod name failover pod name:redis-zgrujr-redis-0 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-redis-zgrujr" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-redis-zgrujr" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-redis-zgrujr" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-0 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-redis-zgrujr.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-redis-zgrujr created apply test-chaos-mesh-oom-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-oom-redis-zgrujr.yaml` check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Updating Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:49 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:50 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-redis-zgrujr" force deleted stresschaos.chaos-mesh.org/test-chaos-mesh-oom-redis-zgrujr patched check failover pod name:redis-zgrujr-redis-1 failover oom Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success cluster configure component_tmp: redis apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: redis-zgrujr-reconfiguring- namespace: ns-cwzhr spec: type: Reconfiguring clusterName: redis-zgrujr force: true reconfigures: - componentName: redis parameters: - key: maxclients value: '10001' check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_redis-zgrujr.yaml` opsrequest.operations.kubeblocks.io/redis-zgrujr-reconfiguring-rsk4h created create test_ops_cluster_redis-zgrujr.yaml Success `rm -rf test_ops_cluster_redis-zgrujr.yaml` check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-reconfiguring-rsk4h ns-cwzhr Reconfiguring redis-zgrujr redis,redis Running -/- Sep 11,2025 20:56 UTC+0800 check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Updating Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:56 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:59 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-reconfiguring-rsk4h ns-cwzhr Reconfiguring redis-zgrujr redis,redis Succeed -/- Sep 11,2025 20:56 UTC+0800 check ops status done ops_status:redis-zgrujr-reconfiguring-rsk4h ns-cwzhr Reconfiguring redis-zgrujr redis,redis Succeed -/- Sep 11,2025 20:56 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations redis-zgrujr-reconfiguring-rsk4h --namespace ns-cwzhr ` opsrequest.operations.kubeblocks.io/redis-zgrujr-reconfiguring-rsk4h patched `kbcli cluster delete-ops --name redis-zgrujr-reconfiguring-rsk4h --force --auto-approve --namespace ns-cwzhr ` OpsRequest redis-zgrujr-reconfiguring-rsk4h deleted component_config:redis check config variables `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. current value_actual: 10001 configure:[maxclients] result actual:[10001] equal expected:[10001] `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success test failover networkpartition check cluster status before cluster-failover-networkpartition check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-redis-zgrujr" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkpartition-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-0 action: partition mode: all target: mode: all selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-1 direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkpartition-redis-zgrujr.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkpartition-redis-zgrujr created apply test-chaos-mesh-networkpartition-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-networkpartition-redis-zgrujr.yaml` networkpartition chaos test waiting 120 seconds check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:56 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:59 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-redis-zgrujr" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-redis-zgrujr" not found check failover pod name failover pod name:redis-zgrujr-redis-0 failover networkpartition Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success test failover fullcpuover check cluster status before cluster-failover-fullcpuover check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpuover-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpuover-redis-zgrujr" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpuover-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-fullcpuover-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-0 mode: all stressors: cpu: workers: 100 load: 100 duration: 2m `kubectl apply -f test-chaos-mesh-fullcpuover-redis-zgrujr.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-fullcpuover-redis-zgrujr created apply test-chaos-mesh-fullcpuover-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-fullcpuover-redis-zgrujr.yaml` fullcpuover chaos test waiting 120 seconds check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:56 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:59 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpuover-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpuover-redis-zgrujr" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpuover-redis-zgrujr" not found check failover pod name failover pod name:redis-zgrujr-redis-1 failover fullcpuover Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success test failover networklossover check cluster status before cluster-failover-networklossover check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-redis-zgrujr" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-1 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-redis-zgrujr.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-redis-zgrujr created apply test-chaos-mesh-networklossover-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-networklossover-redis-zgrujr.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:56 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:59 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-redis-zgrujr" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-redis-zgrujr" not found check failover pod name failover pod name:redis-zgrujr-redis-0 failover networklossover Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success test failover networkdelay check cluster status before cluster-failover-networkdelay check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-redis-zgrujr" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkdelay-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-0 mode: all action: delay delay: latency: 2000ms correlation: '100' jitter: 0ms direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkdelay-redis-zgrujr.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkdelay-redis-zgrujr created apply test-chaos-mesh-networkdelay-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-networkdelay-redis-zgrujr.yaml` networkdelay chaos test waiting 120 seconds check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:56 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:59 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-redis-zgrujr" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-redis-zgrujr" not found check failover pod name failover pod name:redis-zgrujr-redis-0 failover networkdelay Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success test failover dnserror check cluster status before cluster-failover-dnserror check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-redis-zgrujr" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnserror-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-0 mode: all action: error duration: 2m `kubectl apply -f test-chaos-mesh-dnserror-redis-zgrujr.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnserror-redis-zgrujr created apply test-chaos-mesh-dnserror-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-dnserror-redis-zgrujr.yaml` dnserror chaos test waiting 120 seconds check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:56 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:59 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-redis-zgrujr" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-redis-zgrujr" not found check failover pod name failover pod name:redis-zgrujr-redis-0 failover dnserror Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success test failover timeoffset check cluster status before cluster-failover-timeoffset check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-redis-zgrujr" not found Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: TimeChaos metadata: name: test-chaos-mesh-timeoffset-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-0 mode: all timeOffset: '-10m' clockIds: - CLOCK_REALTIME duration: 2m `kubectl apply -f test-chaos-mesh-timeoffset-redis-zgrujr.yaml` timechaos.chaos-mesh.org/test-chaos-mesh-timeoffset-redis-zgrujr created apply test-chaos-mesh-timeoffset-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-timeoffset-redis-zgrujr.yaml` timeoffset chaos test waiting 120 seconds check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:56 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 100m / 100m 512Mi / 512Mi data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:59 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-redis-zgrujr" force deleted Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-redis-zgrujr" not found check failover pod name failover pod name:redis-zgrujr-redis-0 failover timeoffset Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale redis-zgrujr --auto-approve --force=true --components redis --cpu 200m --memory 0.6Gi --namespace ns-cwzhr ` OpsRequest redis-zgrujr-verticalscaling-4h5bg created successfully, you can view the progress: kbcli cluster describe-ops redis-zgrujr-verticalscaling-4h5bg -n ns-cwzhr check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-verticalscaling-4h5bg ns-cwzhr VerticalScaling redis-zgrujr redis Running 0/2 Sep 11,2025 21:14 UTC+0800 check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Updating Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:15 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:14 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-verticalscaling-4h5bg ns-cwzhr VerticalScaling redis-zgrujr redis Succeed 2/2 Sep 11,2025 21:14 UTC+0800 check ops status done ops_status:redis-zgrujr-verticalscaling-4h5bg ns-cwzhr VerticalScaling redis-zgrujr redis Succeed 2/2 Sep 11,2025 21:14 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations redis-zgrujr-verticalscaling-4h5bg --namespace ns-cwzhr ` opsrequest.operations.kubeblocks.io/redis-zgrujr-verticalscaling-4h5bg patched `kbcli cluster delete-ops --name redis-zgrujr-verticalscaling-4h5bg --force --auto-approve --namespace ns-cwzhr ` OpsRequest redis-zgrujr-verticalscaling-4h5bg deleted `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success test failover networkbandwidthover check cluster status before cluster-failover-networkbandwidthover check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-redis-zgrujr" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkbandwidthover-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-1 action: bandwidth mode: all bandwidth: rate: '1bps' limit: 20971520 buffer: 10000 duration: 2m `kubectl apply -f test-chaos-mesh-networkbandwidthover-redis-zgrujr.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkbandwidthover-redis-zgrujr created apply test-chaos-mesh-networkbandwidthover-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-networkbandwidthover-redis-zgrujr.yaml` networkbandwidthover chaos test waiting 120 seconds check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:15 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:14 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-redis-zgrujr" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-redis-zgrujr" not found check failover pod name failover pod name:redis-zgrujr-redis-0 failover networkbandwidthover Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success cluster redis scale-out cluster redis scale-out replicas: 3 check cluster status before ops check cluster status done cluster_status:Running No resources found in redis-zgrujr namespace. `kbcli cluster scale-out redis-zgrujr --auto-approve --force=true --components redis --replicas 1 --namespace ns-cwzhr ` OpsRequest redis-zgrujr-horizontalscaling-mr2c2 created successfully, you can view the progress: kbcli cluster describe-ops redis-zgrujr-horizontalscaling-mr2c2 -n ns-cwzhr check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-horizontalscaling-mr2c2 ns-cwzhr HorizontalScaling redis-zgrujr redis Running 0/1 Sep 11,2025 21:19 UTC+0800 check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Updating Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:15 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:14 UTC+0800 redis-zgrujr-redis-2 ns-cwzhr redis-zgrujr redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:19 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 redis-zgrujr-redis-2 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done No resources found in redis-zgrujr namespace. check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-horizontalscaling-mr2c2 ns-cwzhr HorizontalScaling redis-zgrujr redis Succeed 1/1 Sep 11,2025 21:19 UTC+0800 check ops status done ops_status:redis-zgrujr-horizontalscaling-mr2c2 ns-cwzhr HorizontalScaling redis-zgrujr redis Succeed 1/1 Sep 11,2025 21:19 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations redis-zgrujr-horizontalscaling-mr2c2 --namespace ns-cwzhr ` opsrequest.operations.kubeblocks.io/redis-zgrujr-horizontalscaling-mr2c2 patched `kbcli cluster delete-ops --name redis-zgrujr-horizontalscaling-mr2c2 --force --auto-approve --namespace ns-cwzhr ` OpsRequest redis-zgrujr-horizontalscaling-mr2c2 deleted `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success get redis-sentinel replicas cluster redis scale-in cluster redis scale-in replicas: 2 check cluster status before ops check cluster status done cluster_status:Running No resources found in redis-zgrujr namespace. `kbcli cluster scale-in redis-zgrujr --auto-approve --force=true --components redis --replicas 1 --namespace ns-cwzhr ` OpsRequest redis-zgrujr-horizontalscaling-fg72x created successfully, you can view the progress: kbcli cluster describe-ops redis-zgrujr-horizontalscaling-fg72x -n ns-cwzhr check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-horizontalscaling-fg72x ns-cwzhr HorizontalScaling redis-zgrujr redis Running 0/1 Sep 11,2025 21:20 UTC+0800 check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running primary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:15 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:14 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 20:48 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 20:48 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-0;secondary: redis-zgrujr-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done No resources found in redis-zgrujr namespace. check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-horizontalscaling-fg72x ns-cwzhr HorizontalScaling redis-zgrujr redis Succeed 1/1 Sep 11,2025 21:20 UTC+0800 check ops status done ops_status:redis-zgrujr-horizontalscaling-fg72x ns-cwzhr HorizontalScaling redis-zgrujr redis Succeed 1/1 Sep 11,2025 21:20 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations redis-zgrujr-horizontalscaling-fg72x --namespace ns-cwzhr ` opsrequest.operations.kubeblocks.io/redis-zgrujr-horizontalscaling-fg72x patched `kbcli cluster delete-ops --name redis-zgrujr-horizontalscaling-fg72x --force --auto-approve --namespace ns-cwzhr ` OpsRequest redis-zgrujr-horizontalscaling-fg72x deleted `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart redis-zgrujr --auto-approve --force=true --namespace ns-cwzhr ` OpsRequest redis-zgrujr-restart-d6snw created successfully, you can view the progress: kbcli cluster describe-ops redis-zgrujr-restart-d6snw -n ns-cwzhr check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-restart-d6snw ns-cwzhr Restart redis-zgrujr redis,redis-sentinel Running 0/5 Sep 11,2025 21:21 UTC+0800 check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Updating Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:22 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:22 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:21 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:21 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:21 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-restart-d6snw ns-cwzhr Restart redis-zgrujr redis,redis-sentinel Succeed 5/5 Sep 11,2025 21:21 UTC+0800 check ops status done ops_status:redis-zgrujr-restart-d6snw ns-cwzhr Restart redis-zgrujr redis,redis-sentinel Succeed 5/5 Sep 11,2025 21:21 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations redis-zgrujr-restart-d6snw --namespace ns-cwzhr ` opsrequest.operations.kubeblocks.io/redis-zgrujr-restart-d6snw patched `kbcli cluster delete-ops --name redis-zgrujr-restart-d6snw --force --auto-approve --namespace ns-cwzhr ` OpsRequest redis-zgrujr-restart-d6snw deleted `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success test failover networkduplicate check cluster status before cluster-failover-networkduplicate check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-redis-zgrujr" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-redis-zgrujr" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkduplicate-redis-zgrujr namespace: ns-cwzhr spec: selector: namespaces: - ns-cwzhr labelSelectors: apps.kubeblocks.io/pod-name: redis-zgrujr-redis-1 mode: all action: duplicate duplicate: duplicate: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkduplicate-redis-zgrujr.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkduplicate-redis-zgrujr created apply test-chaos-mesh-networkduplicate-redis-zgrujr.yaml Success `rm -rf test-chaos-mesh-networkduplicate-redis-zgrujr.yaml` networkduplicate chaos test waiting 120 seconds check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:22 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:22 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:21 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:21 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:21 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-redis-zgrujr --namespace ns-cwzhr ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-redis-zgrujr" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-redis-zgrujr" not found check failover pod name failover pod name:redis-zgrujr-redis-1 failover networkduplicate Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success cluster update terminationPolicy WipeOut `kbcli cluster update redis-zgrujr --termination-policy=WipeOut --namespace ns-cwzhr ` cluster.apps.kubeblocks.io/redis-zgrujr updated (no change) check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:22 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:22 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:21 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:21 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:21 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done cluster datafile backup `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.name***"` `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.namespace***"` `kubectl get secrets kb-backuprepo-jtvgz -n kb-ffipo -o jsonpath="***.data.accessKeyId***"` `kubectl get secrets kb-backuprepo-jtvgz -n kb-ffipo -o jsonpath="***.data.secretAccessKey***"` KUBEBLOCKS NAMESPACE:kb-ffipo get kubeblocks namespace done `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-ffipo -o jsonpath="***.items[0].data.root-user***"` `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-ffipo -o jsonpath="***.items[0].data.root-password***"` minio_user:kbclitest,minio_password:kbclitest,minio_endpoint:kbcli-test-minio.kb-ffipo.svc.cluster.local:9000 list minio bucket kbcli-test `echo 'mc config host add minioserver http://kbcli-test-minio.kb-ffipo.svc.cluster.local:9000 kbclitest kbclitest;mc ls minioserver' | kubectl exec -it kbcli-test-minio-cb9fd846b-2rk2d --namespace kb-ffipo -- bash` Unable to use a TTY - input is not a terminal or the right kind of file list minio bucket done default backuprepo:backuprepo-kbcli-test exists `kbcli cluster backup redis-zgrujr --method datafile --namespace ns-cwzhr ` Backup backup-ns-cwzhr-redis-zgrujr-20250911212619 created successfully, you can view the progress: kbcli cluster list-backups --names=backup-ns-cwzhr-redis-zgrujr-20250911212619 -n ns-cwzhr check backup status `kbcli cluster list-backups redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION DELETION-POLICY CREATE-TIME COMPLETION-TIME EXPIRATION backup-ns-cwzhr-redis-zgrujr-20250911212619 ns-cwzhr redis-zgrujr datafile Delete Sep 11,2025 21:26 UTC+0800 backup_status:redis-zgrujr-datafile-Running backup_status:redis-zgrujr-datafile-Running backup_status:redis-zgrujr-datafile-Running check backup status done backup_status:backup-ns-cwzhr-redis-zgrujr-20250911212619 ns-cwzhr redis-zgrujr datafile Completed 13197 16s Delete Sep 11,2025 21:26 UTC+0800 Sep 11,2025 21:26 UTC+0800 cluster restore backup Error from server (NotFound): opsrequests.operations.kubeblocks.io "redis-zgrujr-backup" not found `kbcli cluster describe-backup --names backup-ns-cwzhr-redis-zgrujr-20250911212619 --namespace ns-cwzhr ` Name: backup-ns-cwzhr-redis-zgrujr-20250911212619 Cluster: redis-zgrujr Namespace: ns-cwzhr Spec: Method: datafile Policy Name: redis-zgrujr-redis-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-backup-ns-cwzhr-redis-zgrujr-20250911212619-d74d00f TargetPodName: redis-zgrujr-redis-0 Phase: Completed Start Time: Sep 11,2025 21:26 UTC+0800 Completion Time: Sep 11,2025 21:26 UTC+0800 Status: Phase: Completed Total Size: 13197 ActionSet Name: redis-physical-br Repository: backuprepo-kbcli-test Duration: 16s Start Time: Sep 11,2025 21:26 UTC+0800 Completion Time: Sep 11,2025 21:26 UTC+0800 Path: /ns-cwzhr/redis-zgrujr-f131490d-4f10-4a30-a521-8d1edcdd2533/redis/backup-ns-cwzhr-redis-zgrujr-20250911212619 Warning Events: `kbcli cluster restore redis-zgrujr-backup --backup backup-ns-cwzhr-redis-zgrujr-20250911212619 --namespace ns-cwzhr ` Cluster redis-zgrujr-backup created check cluster status `kbcli cluster list redis-zgrujr-backup --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr-backup ns-cwzhr redis WipeOut Creating Sep 11,2025 21:26 UTC+0800 clusterdefinition.kubeblocks.io/name=redis cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr-backup --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-backup-redis-0 ns-cwzhr redis-zgrujr-backup redis Running primary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:27 UTC+0800 redis-zgrujr-backup-redis-1 ns-cwzhr redis-zgrujr-backup redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:28 UTC+0800 redis-zgrujr-backup-redis-sentinel-0 ns-cwzhr redis-zgrujr-backup redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:26 UTC+0800 redis-zgrujr-backup-redis-sentinel-1 ns-cwzhr redis-zgrujr-backup redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:26 UTC+0800 redis-zgrujr-backup-redis-sentinel-2 ns-cwzhr redis-zgrujr-backup redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:26 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-backup-redis-0;secondary: redis-zgrujr-backup-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr-backup` set secret: redis-zgrujr-backup-redis-account-default `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-backup-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-backup-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kbcli cluster describe-backup --names backup-ns-cwzhr-redis-zgrujr-20250911212619 --namespace ns-cwzhr ` Name: backup-ns-cwzhr-redis-zgrujr-20250911212619 Cluster: redis-zgrujr Namespace: ns-cwzhr Spec: Method: datafile Policy Name: redis-zgrujr-redis-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-backup-ns-cwzhr-redis-zgrujr-20250911212619-d74d00f TargetPodName: redis-zgrujr-redis-0 Phase: Completed Start Time: Sep 11,2025 21:26 UTC+0800 Completion Time: Sep 11,2025 21:26 UTC+0800 Status: Phase: Completed Total Size: 13197 ActionSet Name: redis-physical-br Repository: backuprepo-kbcli-test Duration: 16s Start Time: Sep 11,2025 21:26 UTC+0800 Completion Time: Sep 11,2025 21:26 UTC+0800 Path: /ns-cwzhr/redis-zgrujr-f131490d-4f10-4a30-a521-8d1edcdd2533/redis/backup-ns-cwzhr-redis-zgrujr-20250911212619 Warning Events: cluster connect `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr-backup` set secret: redis-zgrujr-backup-redis-account-default `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. # Server redis_version:6.2.14 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:33f32d76680500a7 redis_mode:standalone os:Linux 5.15.0-1092-azure x86_64 arch_bits:64 monotonic_clock:POSIX clock_gettime multiplexing_api:epoll atomicvar_api:c11-builtin gcc_version:9.4.0 process_id:1 process_supervised:no run_id:a404a25a706cbdbea8c02b592da2a25921febdd7 tcp_port:6379 server_time_usec:1757597343639114 uptime_in_seconds:69 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:12766879 executable:/redis-server config_file:/etc/redis/redis.conf io_threads_active:0 `echo " echo \"CONFIG GET maxclients\" | redis-cli -h redis-zgrujr-backup-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-backup-redis-0 --namespace ns-cwzhr -- bash ` Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. maxclients 10000 connect cluster Success set max_connections to 10000 delete cluster redis-zgrujr-backup `kbcli cluster delete redis-zgrujr-backup --auto-approve --namespace ns-cwzhr ` Cluster redis-zgrujr-backup deleted pod_info:redis-zgrujr-backup-redis-0 3/3 Running 0 88s redis-zgrujr-backup-redis-1 3/3 Running 0 59s redis-zgrujr-backup-redis-sentinel-0 2/2 Running 0 2m27s redis-zgrujr-backup-redis-sentinel-1 2/2 Running 0 2m27s redis-zgrujr-backup-redis-sentinel-2 2/2 Running 0 2m27s No resources found in ns-cwzhr namespace. delete cluster pod done No resources found in ns-cwzhr namespace. check cluster resource non-exist OK: pvc No resources found in ns-cwzhr namespace. delete cluster done No resources found in ns-cwzhr namespace. No resources found in ns-cwzhr namespace. No resources found in ns-cwzhr namespace. cluster rebulid instances apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: redis-zgrujr-rebuildinstance- namespace: ns-cwzhr spec: type: RebuildInstance clusterName: redis-zgrujr force: true rebuildFrom: - componentName: redis instances: - name: redis-zgrujr-redis-0 backupName: backup-ns-cwzhr-redis-zgrujr-20250911212619 inPlace: true check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_redis-zgrujr.yaml` opsrequest.operations.kubeblocks.io/redis-zgrujr-rebuildinstance-t45sz created create test_ops_cluster_redis-zgrujr.yaml Success `rm -rf test_ops_cluster_redis-zgrujr.yaml` check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Running 0/1 Sep 11,2025 21:29 UTC+0800 check ops status done ops_status:redis-zgrujr-rebuildinstance-t45sz ns-cwzhr RebuildInstance redis-zgrujr redis Succeed 1/1 Sep 11,2025 21:29 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations redis-zgrujr-rebuildinstance-t45sz --namespace ns-cwzhr ` opsrequest.operations.kubeblocks.io/redis-zgrujr-rebuildinstance-t45sz patched `kbcli cluster delete-ops --name redis-zgrujr-rebuildinstance-t45sz --force --auto-approve --namespace ns-cwzhr ` OpsRequest redis-zgrujr-rebuildinstance-t45sz deleted check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:29 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:22 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:21 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:21 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:21 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups backup-ns-cwzhr-redis-zgrujr-20250911212619 --namespace ns-cwzhr ` backup.dataprotection.kubeblocks.io/backup-ns-cwzhr-redis-zgrujr-20250911212619 patched `kbcli cluster delete-backup redis-zgrujr --name backup-ns-cwzhr-redis-zgrujr-20250911212619 --force --auto-approve --namespace ns-cwzhr ` Backup backup-ns-cwzhr-redis-zgrujr-20250911212619 deleted No opsrequests found in ns-cwzhr namespace. `kubectl get backupschedule -l app.kubernetes.io/instance=redis-zgrujr ` `kubectl get backupschedule redis-zgrujr-redis-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched check backup status `kbcli cluster list-backups redis-zgrujr --namespace ns-cwzhr ` No backups found in ns-cwzhr namespace. No backups found in ns-cwzhr namespace. backup_status:-- No backups found in ns-cwzhr namespace. backup_status:-- No backups found in ns-cwzhr namespace. backup_status:-- No backups found in ns-cwzhr namespace. backup_status:-- `kubectl get backupschedule -l app.kubernetes.io/instance=redis-zgrujr ` `kubectl get backupschedule redis-zgrujr-redis-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched (no change) backup_status:redis-zgrujr-datafile-Running check backup status done backup_status:redis-zgrujr-datafile-20250911133200 ns-cwzhr redis-zgrujr datafile Completed 13537 10s Delete Sep 11,2025 21:32 UTC+0800 Sep 11,2025 21:32 UTC+0800 Sep 18,2025 21:32 UTC+0800 `kubectl get backupschedule -l app.kubernetes.io/instance=redis-zgrujr ` `kubectl get backupschedule redis-zgrujr-redis-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/redis-zgrujr-redis-backup-schedule patched (no change) cluster restore backup Error from server (NotFound): opsrequests.operations.kubeblocks.io "redis-zgrujr-backup" not found `kbcli cluster describe-backup --names redis-zgrujr-datafile-20250911133200 --namespace ns-cwzhr ` Name: redis-zgrujr-datafile-20250911133200 Cluster: redis-zgrujr Namespace: ns-cwzhr Spec: Method: datafile Policy Name: redis-zgrujr-redis-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-redis-zgrujr-datafile-20250911133200-5b0b00c1 TargetPodName: redis-zgrujr-redis-0 Phase: Completed Start Time: Sep 11,2025 21:32 UTC+0800 Completion Time: Sep 11,2025 21:32 UTC+0800 Status: Phase: Completed Total Size: 13537 ActionSet Name: redis-physical-br Repository: backuprepo-kbcli-test Duration: 10s Expiration Time: Sep 18,2025 21:32 UTC+0800 Start Time: Sep 11,2025 21:32 UTC+0800 Completion Time: Sep 11,2025 21:32 UTC+0800 Path: /ns-cwzhr/redis-zgrujr-f131490d-4f10-4a30-a521-8d1edcdd2533/redis/redis-zgrujr-datafile-20250911133200 Warning Events: `kbcli cluster restore redis-zgrujr-backup --backup redis-zgrujr-datafile-20250911133200 --namespace ns-cwzhr ` Cluster redis-zgrujr-backup created check cluster status `kbcli cluster list redis-zgrujr-backup --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr-backup ns-cwzhr redis WipeOut Creating Sep 11,2025 21:32 UTC+0800 clusterdefinition.kubeblocks.io/name=redis cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr-backup --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-backup-redis-0 ns-cwzhr redis-zgrujr-backup redis Running primary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:33 UTC+0800 redis-zgrujr-backup-redis-1 ns-cwzhr redis-zgrujr-backup redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:34 UTC+0800 redis-zgrujr-backup-redis-sentinel-0 ns-cwzhr redis-zgrujr-backup redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:32 UTC+0800 redis-zgrujr-backup-redis-sentinel-1 ns-cwzhr redis-zgrujr-backup redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:32 UTC+0800 redis-zgrujr-backup-redis-sentinel-2 ns-cwzhr redis-zgrujr-backup redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:32 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-backup-redis-0;secondary: redis-zgrujr-backup-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr-backup` set secret: redis-zgrujr-backup-redis-account-default `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-backup-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-backup-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kbcli cluster describe-backup --names redis-zgrujr-datafile-20250911133200 --namespace ns-cwzhr ` Name: redis-zgrujr-datafile-20250911133200 Cluster: redis-zgrujr Namespace: ns-cwzhr Spec: Method: datafile Policy Name: redis-zgrujr-redis-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-redis-zgrujr-datafile-20250911133200-5b0b00c1 TargetPodName: redis-zgrujr-redis-0 Phase: Completed Start Time: Sep 11,2025 21:32 UTC+0800 Completion Time: Sep 11,2025 21:32 UTC+0800 Status: Phase: Completed Total Size: 13537 ActionSet Name: redis-physical-br Repository: backuprepo-kbcli-test Duration: 10s Expiration Time: Sep 18,2025 21:32 UTC+0800 Start Time: Sep 11,2025 21:32 UTC+0800 Completion Time: Sep 11,2025 21:32 UTC+0800 Path: /ns-cwzhr/redis-zgrujr-f131490d-4f10-4a30-a521-8d1edcdd2533/redis/redis-zgrujr-datafile-20250911133200 Warning Events: cluster connect `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr-backup` set secret: redis-zgrujr-backup-redis-account-default `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. # Server redis_version:6.2.14 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:33f32d76680500a7 redis_mode:standalone os:Linux 5.15.0-1092-azure x86_64 arch_bits:64 monotonic_clock:POSIX clock_gettime multiplexing_api:epoll atomicvar_api:c11-builtin gcc_version:9.4.0 process_id:1 process_supervised:no run_id:63dd3bac0a427a393fd22b4c21f81747023afb0c tcp_port:6379 server_time_usec:1757597716157139 uptime_in_seconds:66 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:12767252 executable:/redis-server config_file:/etc/redis/redis.conf io_threads_active:0 `echo " echo \"CONFIG GET maxclients\" | redis-cli -h redis-zgrujr-backup-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-backup-redis-0 --namespace ns-cwzhr -- bash ` Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. maxclients 10000 connect cluster Success set max_connections to 10000 delete cluster redis-zgrujr-backup `kbcli cluster delete redis-zgrujr-backup --auto-approve --namespace ns-cwzhr ` Cluster redis-zgrujr-backup deleted pod_info:redis-zgrujr-backup-redis-0 3/3 Running 0 118s redis-zgrujr-backup-redis-1 3/3 Running 0 56s redis-zgrujr-backup-redis-sentinel-0 2/2 Running 0 2m59s redis-zgrujr-backup-redis-sentinel-1 2/2 Running 0 2m59s redis-zgrujr-backup-redis-sentinel-2 2/2 Running 0 2m59s No resources found in ns-cwzhr namespace. delete cluster pod done No resources found in ns-cwzhr namespace. check cluster resource non-exist OK: pvc No resources found in ns-cwzhr namespace. delete cluster done No resources found in ns-cwzhr namespace. No resources found in ns-cwzhr namespace. No resources found in ns-cwzhr namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups redis-zgrujr-datafile-20250911133200 --namespace ns-cwzhr ` backup.dataprotection.kubeblocks.io/redis-zgrujr-datafile-20250911133200 patched `kbcli cluster delete-backup redis-zgrujr --name redis-zgrujr-datafile-20250911133200 --force --auto-approve --namespace ns-cwzhr ` Backup redis-zgrujr-datafile-20250911133200 deleted No opsrequests found in ns-cwzhr namespace. check cluster storage class cluster volume-snapshot backup `kbcli cluster backup redis-zgrujr --method volume-snapshot --namespace ns-cwzhr ` Backup backup-ns-cwzhr-redis-zgrujr-20250911213542 created successfully, you can view the progress: kbcli cluster list-backups --names=backup-ns-cwzhr-redis-zgrujr-20250911213542 -n ns-cwzhr check backup status `kbcli cluster list-backups redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION DELETION-POLICY CREATE-TIME COMPLETION-TIME EXPIRATION backup-ns-cwzhr-redis-zgrujr-20250911213542 ns-cwzhr redis-zgrujr volume-snapshot Running Delete Sep 11,2025 21:35 UTC+0800 backup_status:redis-zgrujr-volume-snapshot-Running backup_status:redis-zgrujr-volume-snapshot-Running backup_status:redis-zgrujr-volume-snapshot-Running check backup status done backup_status:backup-ns-cwzhr-redis-zgrujr-20250911213542 ns-cwzhr redis-zgrujr volume-snapshot Completed 6Gi 14s Delete Sep 11,2025 21:35 UTC+0800 Sep 11,2025 21:35 UTC+0800 cluster restore backup Error from server (NotFound): opsrequests.operations.kubeblocks.io "redis-zgrujr-backup" not found `kbcli cluster describe-backup --names backup-ns-cwzhr-redis-zgrujr-20250911213542 --namespace ns-cwzhr ` Name: backup-ns-cwzhr-redis-zgrujr-20250911213542 Cluster: redis-zgrujr Namespace: ns-cwzhr Spec: Method: volume-snapshot Policy Name: redis-zgrujr-redis-backup-policy Actions: createVolumeSnapshot-0: panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x2b40d9f] goroutine 1 [running]: github.com/apecloud/kbcli/pkg/cmd/dataprotection.PrintBackupObjDescribe(0xc000e78480, 0xc0008f6008) /home/runner/work/kbcli/kbcli/pkg/cmd/dataprotection/backup.go:480 +0x4bf github.com/apecloud/kbcli/pkg/cmd/dataprotection.DescribeBackups(0xc000e78480, ***0xc000e9f030?, 0x192f1db?, 0xc000d05b08?***) /home/runner/work/kbcli/kbcli/pkg/cmd/dataprotection/backup.go:458 +0x125 github.com/apecloud/kbcli/pkg/cmd/cluster.describeBackups(0x0?, ***0xc000681e80?, 0x0?, 0x4d0c86e700000000?***) /home/runner/work/kbcli/kbcli/pkg/cmd/cluster/dataprotection.go:204 +0x66 github.com/apecloud/kbcli/pkg/cmd/cluster.NewDescribeBackupCmd.func1(0xc001321508?, ***0xc000681e80, 0x0, 0x4***) /home/runner/work/kbcli/kbcli/pkg/cmd/cluster/dataprotection.go:195 +0xe5 github.com/spf13/cobra.(*Command).execute(0xc001321508, ***0xc000681e40, 0x4, 0x4***) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:989 +0xa91 github.com/spf13/cobra.(*Command).ExecuteC(0xc00029a608) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1117 +0x3ff github.com/spf13/cobra.(*Command).Execute(...) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1041 k8s.io/component-base/cli.run(0xc00029a608) /home/runner/go/pkg/mod/k8s.io/component-base@v0.29.2/cli/run.go:146 +0x290 k8s.io/component-base/cli.RunNoErrOutput(...) /home/runner/go/pkg/mod/k8s.io/component-base@v0.29.2/cli/run.go:84 main.main() /home/runner/work/kbcli/kbcli/cmd/cli/main.go:31 +0x18 `kbcli cluster restore redis-zgrujr-backup --backup backup-ns-cwzhr-redis-zgrujr-20250911213542 --namespace ns-cwzhr ` Cluster redis-zgrujr-backup created check cluster status `kbcli cluster list redis-zgrujr-backup --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr-backup ns-cwzhr redis WipeOut Creating Sep 11,2025 21:36 UTC+0800 clusterdefinition.kubeblocks.io/name=redis cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating [Error] check cluster status timeout --------------------------------------get cluster redis-zgrujr-backup yaml-------------------------------------- `kubectl get cluster redis-zgrujr-backup -o yaml --namespace ns-cwzhr ` apiVersion: apps.kubeblocks.io/v1 kind: Cluster metadata: annotations: kubeblocks.io/crd-api-version: apps.kubeblocks.io/v1 kubeblocks.io/ops-request: '[***"name":"redis-zgrujr-backup","type":"Restore"***]' kubeblocks.io/restore-from-backup: '***"redis":***"doReadyRestoreAfterClusterRunning":"false","name":"backup-ns-cwzhr-redis-zgrujr-20250911213542","namespace":"ns-cwzhr","volumeRestorePolicy":"Parallel"***' creationTimestamp: "2025-09-11T13:36:00Z" finalizers: - cluster.kubeblocks.io/finalizer generation: 1 labels: clusterdefinition.kubeblocks.io/name: redis name: redis-zgrujr-backup namespace: ns-cwzhr resourceVersion: "103831" uid: 2a7e7b75-df74-4f78-9307-451a2f55aa65 spec: clusterDef: redis componentSpecs: - annotations: kubeblocks.io/restart: "2025-09-11T13:21:08Z" componentDef: redis-6-1.0.1 disableExporter: false name: redis replicas: 2 resources: limits: cpu: 200m memory: 644245094400m requests: cpu: 200m memory: 644245094400m serviceVersion: 6.2.14 volumeClaimTemplates: - name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 6Gi - annotations: kubeblocks.io/restart: "2025-09-11T13:21:08Z" componentDef: redis-sentinel-6-1.0.1 name: redis-sentinel replicas: 3 resources: limits: cpu: 100m memory: 512Mi requests: cpu: 100m memory: 512Mi serviceVersion: 6.2.14 volumeClaimTemplates: - name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi terminationPolicy: WipeOut topology: replication status: components: redis: observedGeneration: 1 phase: Creating upToDate: true redis-sentinel: observedGeneration: 1 phase: Running upToDate: true conditions: - lastTransitionTime: "2025-09-11T13:36:00Z" message: 'The operator has started the provisioning of Cluster: redis-zgrujr-backup' observedGeneration: 1 reason: PreCheckSucceed status: "True" type: ProvisioningStarted - lastTransitionTime: "2025-09-11T13:36:00Z" message: Successfully applied for resources observedGeneration: 1 reason: ApplyResourcesSucceed status: "True" type: ApplyResources observedGeneration: 1 phase: Updating ------------------------------------------------------------------------------------------------------------------ --------------------------------------describe cluster redis-zgrujr-backup-------------------------------------- `kubectl describe cluster redis-zgrujr-backup --namespace ns-cwzhr ` Name: redis-zgrujr-backup Namespace: ns-cwzhr Labels: clusterdefinition.kubeblocks.io/name=redis Annotations: kubeblocks.io/crd-api-version: apps.kubeblocks.io/v1 kubeblocks.io/ops-request: [***"name":"redis-zgrujr-backup","type":"Restore"***] kubeblocks.io/restore-from-backup: ***"redis":***"doReadyRestoreAfterClusterRunning":"false","name":"backup-ns-cwzhr-redis-zgrujr-20250911213542","namespace":"ns-cwzhr","volumeR... API Version: apps.kubeblocks.io/v1 Kind: Cluster Metadata: Creation Timestamp: 2025-09-11T13:36:00Z Finalizers: cluster.kubeblocks.io/finalizer Generation: 1 Resource Version: 103831 UID: 2a7e7b75-df74-4f78-9307-451a2f55aa65 Spec: Cluster Def: redis Component Specs: Annotations: kubeblocks.io/restart: 2025-09-11T13:21:08Z Component Def: redis-6-1.0.1 Disable Exporter: false Name: redis Replicas: 2 Resources: Limits: Cpu: 200m Memory: 644245094400m Requests: Cpu: 200m Memory: 644245094400m Service Version: 6.2.14 Volume Claim Templates: Name: data Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 6Gi Annotations: kubeblocks.io/restart: 2025-09-11T13:21:08Z Component Def: redis-sentinel-6-1.0.1 Name: redis-sentinel Replicas: 3 Resources: Limits: Cpu: 100m Memory: 512Mi Requests: Cpu: 100m Memory: 512Mi Service Version: 6.2.14 Volume Claim Templates: Name: data Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 1Gi Termination Policy: WipeOut Topology: replication Status: Components: Redis: Observed Generation: 1 Phase: Creating Up To Date: true Redis - Sentinel: Observed Generation: 1 Phase: Running Up To Date: true Conditions: Last Transition Time: 2025-09-11T13:36:00Z Message: The operator has started the provisioning of Cluster: redis-zgrujr-backup Observed Generation: 1 Reason: PreCheckSucceed Status: True Type: ProvisioningStarted Last Transition Time: 2025-09-11T13:36:00Z Message: Successfully applied for resources Observed Generation: 1 Reason: ApplyResourcesSucceed Status: True Type: ApplyResources Observed Generation: 1 Phase: Updating Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal PreCheckSucceed 7m24s cluster-controller The operator has started the provisioning of Cluster: redis-zgrujr-backup Normal ApplyResourcesSucceed 7m24s cluster-controller Successfully applied for resources Normal ClusterComponentPhaseTransition 6m9s (x6 over 7m22s) cluster-controller cluster component redis-sentinel is Creating Normal ClusterComponentPhaseTransition 4m47s (x10 over 6m6s) cluster-controller cluster component redis-sentinel is Running Normal ClusterComponentPhaseTransition 4m47s (x3 over 6m4s) cluster-controller cluster component redis is Creating ------------------------------------------------------------------------------------------------------------------ check pod status `kbcli cluster list-instances redis-zgrujr-backup --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-backup-redis-0 ns-cwzhr redis-zgrujr-backup redis Running primary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:37 UTC+0800 redis-zgrujr-backup-redis-1 ns-cwzhr redis-zgrujr-backup redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:42 UTC+0800 redis-zgrujr-backup-redis-sentinel-0 ns-cwzhr redis-zgrujr-backup redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:36 UTC+0800 redis-zgrujr-backup-redis-sentinel-1 ns-cwzhr redis-zgrujr-backup redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:36 UTC+0800 redis-zgrujr-backup-redis-sentinel-2 ns-cwzhr redis-zgrujr-backup redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:36 UTC+0800 check pod status done check cluster status again cluster_status:Updating check cluster status again done check cluster role check cluster role done primary: redis-zgrujr-backup-redis-0;secondary: redis-zgrujr-backup-redis-1 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr-backup` set secret: redis-zgrujr-backup-redis-account-default `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-backup-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-backup-redis-0 --namespace ns-cwzhr -- bash` check cluster connect done `kbcli cluster describe-backup --names backup-ns-cwzhr-redis-zgrujr-20250911213542 --namespace ns-cwzhr ` Name: backup-ns-cwzhr-redis-zgrujr-20250911213542 Cluster: redis-zgrujr Namespace: ns-cwzhr Spec: Method: volume-snapshot Policy Name: redis-zgrujr-redis-backup-policy Actions: createVolumeSnapshot-0: panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x2b40d9f] goroutine 1 [running]: github.com/apecloud/kbcli/pkg/cmd/dataprotection.PrintBackupObjDescribe(0xc0013e2300, 0xc000ad2588) /home/runner/work/kbcli/kbcli/pkg/cmd/dataprotection/backup.go:480 +0x4bf github.com/apecloud/kbcli/pkg/cmd/dataprotection.DescribeBackups(0xc0013e2300, ***0xc0010f9320?, 0x192f1db?, 0xc00156a008?***) /home/runner/work/kbcli/kbcli/pkg/cmd/dataprotection/backup.go:458 +0x125 github.com/apecloud/kbcli/pkg/cmd/cluster.describeBackups(0x0?, ***0xc0006d02c0?, 0x0?, 0xdbc293cc00000000?***) /home/runner/work/kbcli/kbcli/pkg/cmd/cluster/dataprotection.go:204 +0x66 github.com/apecloud/kbcli/pkg/cmd/cluster.NewDescribeBackupCmd.func1(0xc000d0fb08?, ***0xc0006d02c0, 0x0, 0x4***) /home/runner/work/kbcli/kbcli/pkg/cmd/cluster/dataprotection.go:195 +0xe5 github.com/spf13/cobra.(*Command).execute(0xc000d0fb08, ***0xc0006d0200, 0x4, 0x4***) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:989 +0xa91 github.com/spf13/cobra.(*Command).ExecuteC(0xc000abe908) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1117 +0x3ff github.com/spf13/cobra.(*Command).Execute(...) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1041 k8s.io/component-base/cli.run(0xc000abe908) /home/runner/go/pkg/mod/k8s.io/component-base@v0.29.2/cli/run.go:146 +0x290 k8s.io/component-base/cli.RunNoErrOutput(...) /home/runner/go/pkg/mod/k8s.io/component-base@v0.29.2/cli/run.go:84 main.main() /home/runner/work/kbcli/kbcli/cmd/cli/main.go:31 +0x18 cluster connect `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr-backup` set secret: redis-zgrujr-backup-redis-account-default `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-backup-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. # Server redis_version:6.2.14 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:33f32d76680500a7 redis_mode:standalone os:Linux 5.15.0-1092-azure x86_64 arch_bits:64 monotonic_clock:POSIX clock_gettime multiplexing_api:epoll atomicvar_api:c11-builtin gcc_version:9.4.0 process_id:1 process_supervised:no run_id:ae7ab18904d1e9d75fe51782f18d7fb71e703b74 tcp_port:6379 server_time_usec:1757598215715425 uptime_in_seconds:78 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:12767751 executable:/redis-server config_file:/etc/redis/redis.conf io_threads_active:0 `echo " echo \"CONFIG GET maxclients\" | redis-cli -h redis-zgrujr-backup-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-backup-redis-0 --namespace ns-cwzhr -- bash ` Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. maxclients 10000 connect cluster Success set max_connections to 10000 delete cluster redis-zgrujr-backup `kbcli cluster delete redis-zgrujr-backup --auto-approve --namespace ns-cwzhr ` Cluster redis-zgrujr-backup deleted pod_info:redis-zgrujr-backup-redis-0 3/3 Running 0 6m18s redis-zgrujr-backup-redis-1 3/3 Running 0 68s redis-zgrujr-backup-redis-sentinel-0 2/2 Running 0 7m36s redis-zgrujr-backup-redis-sentinel-1 2/2 Running 0 7m36s redis-zgrujr-backup-redis-sentinel-2 2/2 Running 0 7m36s No resources found in ns-cwzhr namespace. delete cluster pod done No resources found in ns-cwzhr namespace. check cluster resource non-exist OK: pvc No resources found in ns-cwzhr namespace. delete cluster done No resources found in ns-cwzhr namespace. No resources found in ns-cwzhr namespace. No resources found in ns-cwzhr namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups backup-ns-cwzhr-redis-zgrujr-20250911213542 --namespace ns-cwzhr ` backup.dataprotection.kubeblocks.io/backup-ns-cwzhr-redis-zgrujr-20250911213542 patched `kbcli cluster delete-backup redis-zgrujr --name backup-ns-cwzhr-redis-zgrujr-20250911213542 --force --auto-approve --namespace ns-cwzhr ` Backup backup-ns-cwzhr-redis-zgrujr-20250911213542 deleted No opsrequests found in ns-cwzhr namespace. cluster list-logs `kbcli cluster list-logs redis-zgrujr --namespace ns-cwzhr ` No log files found. Error from server (NotFound): pods "redis-zgrujr-redis-1" not found cluster logs `kbcli cluster logs redis-zgrujr --tail 30 --namespace ns-cwzhr ` Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) + sed '/user kbreplicator on/d' /data/users.acl + mv /data/users.acl.bak /data/users.acl + sed '/user kbreplicator-sentinel on/d' /data/users.acl + mv /data/users.acl.bak /data/users.acl + build_redis_default_accounts + unset_xtrace_when_ut_mode_false + '[' false == false ']' + set +x + echo 'aclfile /data/users.acl' + echo 'build default accounts succeeded!' build default accounts succeeded! + start_redis_server + module_path=/opt/redis-stack/lib + [[ '' == \t\r\u\e ]] + exec_cmd='exec redis-server /etc/redis/redis.conf' + '[' -f /opt/redis-stack/lib/redisearch.so ']' + exec_cmd='exec redis-server /etc/redis/redis.conf --loadmodule /opt/redis-stack/lib/redisearch.so ' + '[' -f /opt/redis-stack/lib/redistimeseries.so ']' + exec_cmd='exec redis-server /etc/redis/redis.conf --loadmodule /opt/redis-stack/lib/redisearch.so --loadmodule /opt/redis-stack/lib/redistimeseries.so ' + '[' -f /opt/redis-stack/lib/rejson.so ']' + exec_cmd='exec redis-server /etc/redis/redis.conf --loadmodule /opt/redis-stack/lib/redisearch.so --loadmodule /opt/redis-stack/lib/redistimeseries.so --loadmodule /opt/redis-stack/lib/rejson.so ' + '[' -f /opt/redis-stack/lib/redisbloom.so ']' + exec_cmd='exec redis-server /etc/redis/redis.conf --loadmodule /opt/redis-stack/lib/redisearch.so --loadmodule /opt/redis-stack/lib/redistimeseries.so --loadmodule /opt/redis-stack/lib/rejson.so --loadmodule /opt/redis-stack/lib/redisbloom.so ' + '[' -f /opt/redis-stack/lib/redisgraph.so ']' + exec_cmd='exec redis-server /etc/redis/redis.conf --loadmodule /opt/redis-stack/lib/redisearch.so --loadmodule /opt/redis-stack/lib/redistimeseries.so --loadmodule /opt/redis-stack/lib/rejson.so --loadmodule /opt/redis-stack/lib/redisbloom.so --loadmodule /opt/redis-stack/lib/redisgraph.so ' + '[' -f /opt/redis-stack/lib/rediscompat.so ']' Starting redis server cmd: exec redis-server /etc/redis/redis.conf --loadmodule /opt/redis-stack/lib/redisearch.so --loadmodule /opt/redis-stack/lib/redistimeseries.so --loadmodule /opt/redis-stack/lib/rejson.so --loadmodule /opt/redis-stack/lib/redisbloom.so --loadmodule /opt/redis-stack/lib/redisgraph.so + echo 'Starting redis server cmd: exec redis-server /etc/redis/redis.conf --loadmodule /opt/redis-stack/lib/redisearch.so --loadmodule /opt/redis-stack/lib/redistimeseries.so --loadmodule /opt/redis-stack/lib/rejson.so --loadmodule /opt/redis-stack/lib/redisbloom.so --loadmodule /opt/redis-stack/lib/redisgraph.so ' + eval 'exec redis-server /etc/redis/redis.conf --loadmodule /opt/redis-stack/lib/redisearch.so --loadmodule /opt/redis-stack/lib/redistimeseries.so --loadmodule /opt/redis-stack/lib/rejson.so --loadmodule /opt/redis-stack/lib/redisbloom.so --loadmodule /opt/redis-stack/lib/redisgraph.so ' ++ exec redis-server /etc/redis/redis.conf --loadmodule /opt/redis-stack/lib/redisearch.so --loadmodule /opt/redis-stack/lib/redistimeseries.so --loadmodule /opt/redis-stack/lib/rejson.so --loadmodule /opt/redis-stack/lib/redisbloom.so --loadmodule /opt/redis-stack/lib/redisgraph.so cluster logs running `kbcli cluster logs redis-zgrujr --tail 30 --file-type=running --namespace ns-cwzhr ` 65:C 11 Sep 2025 13:22:29.741 * AOF rewrite: 0 MB of memory used by copy-on-write 1:S 11 Sep 2025 13:22:29.768 * Background AOF rewrite terminated with success 1:S 11 Sep 2025 13:22:29.769 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB) 1:S 11 Sep 2025 13:22:29.769 * Background AOF rewrite finished successfully 1:S 11 Sep 2025 13:22:49.239 # Connection with master lost. 1:S 11 Sep 2025 13:22:49.239 * Caching the disconnected master state. 1:S 11 Sep 2025 13:22:49.239 * Reconnecting to MASTER redis-zgrujr-redis-0.redis-zgrujr-redis-headless.ns-cwzhr.svc.cluster.local:6379 1:S 11 Sep 2025 13:22:49.242 * MASTER <-> REPLICA sync started 1:S 11 Sep 2025 13:22:49.243 # Error condition on socket for SYNC: Connection refused 1:M 11 Sep 2025 13:22:49.357 * Discarding previously cached master state. 1:M 11 Sep 2025 13:22:49.357 # Setting secondary replication ID to 3427faf087cf1a4eb8c7af098ec6ba8ec7c5b395, valid up to offset: 1209200. New replication ID is 7be5f4d3f724aace96a1e0e3d7251ac5df8caa34 1:M 11 Sep 2025 13:22:49.357 * MASTER MODE enabled (user request from 'id=15 addr=10.244.4.34:52382 laddr=10.244.2.80:6379 fd=8 name=sentinel-e6f8fc0b-cmd age=8 idle=0 flags=x db=0 sub=0 psub=0 multi=4 qbuf=188 qbuf-free=40766 argv-mem=4 obl=45 oll=0 omem=0 tot-mem=61468 events=r cmd=exec user=kbreplicator-sentinel redir=-1') 1:M 11 Sep 2025 13:22:49.364 # CONFIG REWRITE executed with success. 1:M 11 Sep 2025 13:22:57.152 * Replica redis-zgrujr-redis-0.redis-zgrujr-redis-headless.ns-cwzhr.svc.cluster.local:6379 asks for synchronization 1:M 11 Sep 2025 13:22:57.152 * Full resync requested by replica redis-zgrujr-redis-0.redis-zgrujr-redis-headless.ns-cwzhr.svc.cluster.local:6379 1:M 11 Sep 2025 13:22:57.152 * Starting BGSAVE for SYNC with target: disk 1:M 11 Sep 2025 13:22:57.153 * Background saving started by pid 99 99:C 11 Sep 2025 13:22:57.162 * DB saved on disk 99:C 11 Sep 2025 13:22:57.163 * RDB: 0 MB of memory used by copy-on-write 1:M 11 Sep 2025 13:22:57.183 * Background saving terminated with success 1:M 11 Sep 2025 13:22:57.183 * Synchronization with replica redis-zgrujr-redis-0.redis-zgrujr-redis-headless.ns-cwzhr.svc.cluster.local:6379 succeeded 1:M 11 Sep 2025 13:29:56.959 # Connection with replica redis-zgrujr-redis-0.redis-zgrujr-redis-headless.ns-cwzhr.svc.cluster.local:6379 lost. 1:M 11 Sep 2025 13:30:30.528 * Replica redis-zgrujr-redis-0.redis-zgrujr-redis-headless.ns-cwzhr.svc.cluster.local:6379 asks for synchronization 1:M 11 Sep 2025 13:30:30.528 * Full resync requested by replica redis-zgrujr-redis-0.redis-zgrujr-redis-headless.ns-cwzhr.svc.cluster.local:6379 1:M 11 Sep 2025 13:30:30.528 * Starting BGSAVE for SYNC with target: disk 1:M 11 Sep 2025 13:30:30.529 * Background saving started by pid 879 879:C 11 Sep 2025 13:30:30.536 * DB saved on disk 879:C 11 Sep 2025 13:30:30.537 * RDB: 0 MB of memory used by copy-on-write 1:M 11 Sep 2025 13:30:30.611 * Background saving terminated with success 1:M 11 Sep 2025 13:30:30.611 * Synchronization with replica redis-zgrujr-redis-0.redis-zgrujr-redis-headless.ns-cwzhr.svc.cluster.local:6379 succeeded LB_TYPE is set to: intranet cluster expose check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster expose redis-zgrujr --auto-approve --force=true --type intranet --enable false --components redis --role-selector primary --namespace ns-cwzhr ` OpsRequest redis-zgrujr-expose-tb5cj created successfully, you can view the progress: kbcli cluster describe-ops redis-zgrujr-expose-tb5cj -n ns-cwzhr check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-expose-tb5cj ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 21:44 UTC+0800 check cluster status `kbcli cluster list redis-zgrujr --show-labels --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS redis-zgrujr ns-cwzhr redis WipeOut Running Sep 11,2025 20:24 UTC+0800 app.kubernetes.io/instance=redis-zgrujr,clusterdefinition.kubeblocks.io/name=redis check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances redis-zgrujr --namespace ns-cwzhr ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME redis-zgrujr-redis-0 ns-cwzhr redis-zgrujr redis Running secondary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:29 UTC+0800 redis-zgrujr-redis-1 ns-cwzhr redis-zgrujr redis Running primary 0 200m / 200m 644245094400m / 644245094400m data:6Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:22 UTC+0800 redis-zgrujr-redis-sentinel-0 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000001/10.224.0.5 Sep 11,2025 21:21 UTC+0800 redis-zgrujr-redis-sentinel-1 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000002/10.224.0.7 Sep 11,2025 21:21 UTC+0800 redis-zgrujr-redis-sentinel-2 ns-cwzhr redis-zgrujr redis-sentinel Running 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-42425418-vmss000000/10.224.0.9 Sep 11,2025 21:21 UTC+0800 check pod status done check cluster role check cluster role done primary: redis-zgrujr-redis-1;secondary: redis-zgrujr-redis-0 `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check cluster connect `echo 'echo "" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC' | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash` check cluster connect done check ops status `kbcli cluster list-ops redis-zgrujr --status all --namespace ns-cwzhr ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME redis-zgrujr-expose-tb5cj ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 21:44 UTC+0800 ops_status:redis-zgrujr-expose-tb5cj ns-cwzhr Expose redis-zgrujr redis Running 0/1 Sep 11,2025 21:44 UTC+0800 check ops status done ops_status:redis-zgrujr-expose-tb5cj ns-cwzhr Expose redis-zgrujr redis Succeed 1/1 Sep 11,2025 21:44 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations redis-zgrujr-expose-tb5cj --namespace ns-cwzhr ` opsrequest.operations.kubeblocks.io/redis-zgrujr-expose-tb5cj patched `kbcli cluster delete-ops --name redis-zgrujr-expose-tb5cj --force --auto-approve --namespace ns-cwzhr ` OpsRequest redis-zgrujr-expose-tb5cj deleted `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 `echo "echo \"get mykey\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly data: Defaulted container "redis" out of: redis, kbagent, config-manager, init-dbctl (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. nlbzw check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h redis-zgrujr-redis-redis.ns-cwzhr.svc.cluster.local -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-1 --namespace ns-cwzhr -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=redis-zgrujr` set secret: redis-zgrujr-redis-account-default `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.username***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.password***"` `kubectl get secrets redis-zgrujr-redis-account-default -o jsonpath="***.data.port***"` DB_USERNAME_PROXY:;DB_PASSWORD_PROXY:;DB_PORT_PROXY:26379 DB_USERNAME:default;DB_PASSWORD:1A26G0F9bC;DB_PORT:6379;DB_DATABASE:db0 check readonly db_client batch data count `echo "echo \"get executions_loop_key\" | redis-cli -h 127.0.0.1 -p 6379 -a 1A26G0F9bC " | kubectl exec -it redis-zgrujr-redis-0 --namespace ns-cwzhr -- bash ` check readonly db_client batch data Success delete cluster redis-zgrujr `kbcli cluster delete redis-zgrujr --auto-approve --namespace ns-cwzhr ` Cluster redis-zgrujr deleted pod_info:redis-zgrujr-redis-0 3/3 Running 0 14m redis-zgrujr-redis-1 3/3 Running 0 22m redis-zgrujr-redis-sentinel-0 2/2 Running 0 22m redis-zgrujr-redis-sentinel-1 2/2 Running 0 23m redis-zgrujr-redis-sentinel-2 2/2 Running 0 23m No resources found in ns-cwzhr namespace. delete cluster pod done No resources found in ns-cwzhr namespace. check cluster resource non-exist OK: pvc No resources found in ns-cwzhr namespace. delete cluster done No resources found in ns-cwzhr namespace. No resources found in ns-cwzhr namespace. No resources found in ns-cwzhr namespace. Redis Test Suite All Done! Test Engine: redis Test Type: 5 --------------------------------------Redis (Topology = replication Replicas 2) Test Result-------------------------------------- [PASSED]|[Create]|[ComponentDefinition=redis-6-1.0.1;ComponentVersion=redis;ServiceVersion=6.2.14;]|[Description=Create a cluster with the specified component definition redis-6-1.0.1 and component version redis and service version 6.2.14] [PASSED]|[Connect]|[ComponentName=redis]|[Description=Connect to the cluster] [PASSED]|[AddData]|[Values=nlbzw]|[Description=Add data to the cluster] [PASSED]|[CheckAddDataReadonly]|[Values=nlbzw;Role=Readonly]|[Description=Add data to the cluster readonly] [PASSED]|[Expose]|[Enable=true;TYPE=intranet;ComponentName=redis]|[Description=Expose Enable the intranet service with redis component] [PASSED]|[No-Failover]|[HA=Connection Stress;ComponentName=redis]|[Description=Simulates conditions where pods experience connection stress either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Connection load.] [PASSED]|[Failover]|[HA=Pod Kill;ComponentName=redis]|[Description=Simulates conditions where pods experience kill for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to kill.] [PASSED]|[Bench]|[ComponentName=redis]|[Description=Bench the cluster service with redis component] [PASSED]|[Bench]|[HostType=LB;ComponentName=redis]|[Description=Bench the cluster LB service with redis component] [PASSED]|[Failover]|[HA=Network Corrupt;Durations=2m;ComponentName=redis]|[Description=Simulates network corrupt fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to corrupt network.] [PASSED]|[Update]|[Monitor=true]|[Description=Update the cluster Monitor enable] [PASSED]|[Failover]|[HA=Kill 1;ComponentName=redis]|[Description=Simulates conditions where process 1 killed either due to expected/undesired processes thereby testing the application's resilience to unavailability of some replicas due to abnormal termination signals.] [PASSED]|[No-Failover]|[HA=DNS Random;Durations=2m;ComponentName=redis]|[Description=Simulates conditions where pods experience random IP addresses being returned by the DNS service for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to the DNS service returning random IP addresses.] [PASSED]|[VolumeExpansion]|[ComponentName=redis]|[Description=VolumeExpansion the cluster specify component redis] [PASSED]|[Stop]|[-]|[Description=Stop the cluster] [PASSED]|[Start]|[-]|[Description=Start the cluster] [PASSED]|[Failover]|[HA=Pod Failure;Durations=2m;ComponentName=redis]|[Description=Simulates conditions where pods experience failure for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to failure.] [PASSED]|[Failover]|[HA=OOM;Durations=2m;ComponentName=redis]|[Description=Simulates conditions where pods experience OOM either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Memory load.] [PASSED]|[Reconfiguring]|[ComponentName=redis;maxclients=10001]|[Description=Reconfiguring the cluster specify component redis set maxclients=10001] [PASSED]|[No-Failover]|[HA=Network Partition;Durations=2m;ComponentName=redis]|[Description=Simulates network partition fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to partition network.] [PASSED]|[Failover]|[HA=Full CPU;Durations=2m;ComponentName=redis]|[Description=Simulates conditions where pods experience CPU full either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high CPU load.] [PASSED]|[Failover]|[HA=Network Loss;Durations=2m;ComponentName=redis]|[Description=Simulates network loss fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to loss network.] [PASSED]|[No-Failover]|[HA=Network Delay;Durations=2m;ComponentName=redis]|[Description=Simulates network delay fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to delay network.] [PASSED]|[No-Failover]|[HA=DNS Error;Durations=2m;ComponentName=redis]|[Description=Simulates conditions where pods experience DNS service errors for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to DNS service errors.] [PASSED]|[No-Failover]|[HA=Time Offset;Durations=2m;ComponentName=redis]|[Description=Simulates a time offset scenario thereby testing the application's resilience to potential slowness/unavailability of some replicas due to time offset.] [PASSED]|[VerticalScaling]|[ComponentName=redis]|[Description=VerticalScaling the cluster specify component redis] [PASSED]|[Failover]|[HA=Network Bandwidth;Durations=2m;ComponentName=redis]|[Description=Simulates network bandwidth fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to bandwidth network.] [PASSED]|[HorizontalScaling Out]|[ComponentName=redis]|[Description=HorizontalScaling Out the cluster specify component redis] [PASSED]|[HorizontalScaling In]|[ComponentName=redis]|[Description=HorizontalScaling In the cluster specify component redis] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [PASSED]|[No-Failover]|[HA=Network Duplicate;Durations=2m;ComponentName=redis]|[Description=Simulates network duplicate fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to duplicate network.] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Backup]|[BackupMethod=datafile]|[Description=The cluster datafile Backup] [PASSED]|[Restore]|[BackupMethod=datafile]|[Description=The cluster datafile Restore] [PASSED]|[Connect]|[ComponentName=redis]|[Description=Connect to the cluster] [PASSED]|[Delete Restore Cluster]|[BackupMethod=datafile]|[Description=Delete the datafile restore cluster] [PASSED]|[RebuildInstance]|[ComponentName=redis]|[Description=Rebuild the cluster instance specify component redis] [PASSED]|[Backup]|[Schedule=true;BackupMethod=datafile]|[Description=The cluster Schedule datafile Backup] [PASSED]|[Restore]|[Schedule=true;BackupMethod=datafile]|[Description=The cluster Schedule datafile Restore] [PASSED]|[Connect]|[ComponentName=redis]|[Description=Connect to the cluster] [PASSED]|[Delete Restore Cluster]|[Schedule=true;BackupMethod=datafile]|[Description=Delete the Schedule datafile restore cluster] [PASSED]|[Backup]|[BackupMethod=volume-snapshot]|[Description=The cluster volume-snapshot Backup] [PASSED]|[Restore]|[BackupMethod=volume-snapshot]|[Description=The cluster volume-snapshot Restore] [PASSED]|[Connect]|[ComponentName=redis]|[Description=Connect to the cluster] [PASSED]|[Delete Restore Cluster]|[BackupMethod=volume-snapshot]|[Description=Delete the volume-snapshot restore cluster] [PASSED]|[Expose]|[Disable=true;TYPE=intranet;ComponentName=redis]|[Description=Expose Disable the intranet service with redis component] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]