bash test/kbcli/test_kbcli_0.9.sh --type 56 --version 0.9.5 --generate-output true --chaos-mesh true --drain-node true --random-namespace true --region eastus --cloud-provider aks CURRENT_TEST_DIR:test/kbcli source commons files source engines files source kubeblocks files `kubectl get namespace | grep ns-ybzhw ` `kubectl create namespace ns-ybzhw` namespace/ns-ybzhw created create namespace ns-ybzhw done download kbcli `gh release list --repo apecloud/kbcli --limit 100 | (grep "0.9" || true)` `curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v0.9.5-beta.8` Your system is linux_amd64 Installing kbcli ... Downloading ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 32.1M 100 32.1M 0 0 87.8M 0 --:--:-- --:--:-- --:--:-- 87.8M kbcli installed successfully. Kubernetes: v1.32.6 KubeBlocks: 0.9.5 kbcli: 0.9.5-beta.8 WARNING: version difference between kbcli (0.9.5-beta.8) and kubeblocks (0.9.5) Make sure your docker service is running and begin your journey with kbcli: kbcli playground init For more information on how to get started, please visit: https://kubeblocks.io download kbcli v0.9.5-beta.8 done Kubernetes: v1.32.6 KubeBlocks: 0.9.5 kbcli: 0.9.5-beta.8 WARNING: version difference between kbcli (0.9.5-beta.8) and kubeblocks (0.9.5) Kubernetes Env: v1.32.6 POD_RESOURCES: aks kb-default-sc found aks default-vsc found found default storage class: default kubeblocks version is:0.9.5 skip upgrade kubeblocks Error: no repositories to show helm repo add chaos-mesh https://charts.chaos-mesh.org "chaos-mesh" has been added to your repositories add helm chart repo chaos-mesh success chaos mesh already installed check cluster definition set component name:gaussdb set component version set component version:gaussdb set service versions:2.23.1 set service versions sorted:2.23.1 no cluster version found set gaussdb component definition set gaussdb component definition gaussdb-repl REPORT_COUNT 0:0 set replicas first:3,2.23.1 set replicas third:3,2.23.1 set replicas fourth:3,2.23.1 set minimum cmpv service version set minimum cmpv service version replicas:3,2.23.1 REPORT_COUNT:1 CLUSTER_TOPOLOGY:replication topology replication found in cluster definition gaussdb LIMIT_CPU:0.5 LIMIT_MEMORY:2 storage size: 20 No resources found in ns-ybzhw namespace. termination_policy:WipeOut create 3 replica WipeOut gaussdb cluster check cluster definition check component definition set component definition by component version check cmpd by labels set component definition2: gaussdb-repl by component version:gaussdb apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: name: gaussdb-bpsmmq namespace: ns-ybzhw spec: clusterDefinitionRef: gaussdb topology: replication terminationPolicy: WipeOut componentSpecs: - name: gaussdb serviceVersion: 2.23.1 replicas: 3 resources: requests: cpu: 500m memory: 2Gi limits: cpu: 500m memory: 2Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi `kubectl apply -f test_create_gaussdb-bpsmmq.yaml` cluster.apps.kubeblocks.io/gaussdb-bpsmmq created apply test_create_gaussdb-bpsmmq.yaml Success `rm -rf test_create_gaussdb-bpsmmq.yaml` check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Sep 01,2025 11:24 UTC+0800 clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-2;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-1 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-2 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check pod gaussdb-bpsmmq-gaussdb-2 container_name gaussdb exist password 511qH9i3MJ6g check pod gaussdb-bpsmmq-gaussdb-2 container_name exporter exist password 511qH9i3MJ6g check pod gaussdb-bpsmmq-gaussdb-2 container_name lorry exist password 511qH9i3MJ6g check pod gaussdb-bpsmmq-gaussdb-2 container_name config-manager exist password 511qH9i3MJ6g No container logs contain secret password. describe cluster `kbcli cluster describe gaussdb-bpsmmq --namespace ns-ybzhw ` Name: gaussdb-bpsmmq Created Time: Sep 01,2025 11:24 UTC+0800 NAMESPACE CLUSTER-DEFINITION VERSION STATUS TERMINATION-POLICY ns-ybzhw gaussdb Running WipeOut Endpoints: COMPONENT MODE INTERNAL EXTERNAL gaussdb ReadWrite gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:8999 Topology: COMPONENT INSTANCE ROLE STATUS AZ NODE CREATED-TIME gaussdb gaussdb-bpsmmq-gaussdb-0 secondary Running 0 aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb gaussdb-bpsmmq-gaussdb-1 secondary Running 0 aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb gaussdb-bpsmmq-gaussdb-2 primary Running 0 aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 Resources Allocation: COMPONENT DEDICATED CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS gaussdb false 500m / 500m 2Gi / 2Gi data:20Gi default Images: COMPONENT TYPE IMAGE gaussdb docker.io/apecloud/gaussdb:2.23.01.200 Data Protection: BACKUP-REPO AUTO-BACKUP BACKUP-SCHEDULE BACKUP-METHOD BACKUP-RETENTION RECOVERABLE-TIME Show cluster events: kbcli cluster list-events -n ns-ybzhw gaussdb-bpsmmq `kbcli cluster label gaussdb-bpsmmq app.kubernetes.io/instance- --namespace ns-ybzhw ` label "app.kubernetes.io/instance" not found. `kbcli cluster label gaussdb-bpsmmq app.kubernetes.io/instance=gaussdb-bpsmmq --namespace ns-ybzhw ` `kbcli cluster label gaussdb-bpsmmq --list --namespace ns-ybzhw ` NAME NAMESPACE LABELS gaussdb-bpsmmq ns-ybzhw app.kubernetes.io/instance=gaussdb-bpsmmq clusterdefinition.kubeblocks.io/name=gaussdb clusterversion.kubeblocks.io/name= label cluster app.kubernetes.io/instance=gaussdb-bpsmmq Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=gaussdb-bpsmmq --namespace ns-ybzhw ` `kbcli cluster label gaussdb-bpsmmq --list --namespace ns-ybzhw ` NAME NAMESPACE LABELS gaussdb-bpsmmq ns-ybzhw app.kubernetes.io/instance=gaussdb-bpsmmq case.name=kbcli.test1 clusterdefinition.kubeblocks.io/name=gaussdb clusterversion.kubeblocks.io/name= label cluster case.name=kbcli.test1 Success `kbcli cluster label gaussdb-bpsmmq case.name=kbcli.test2 --overwrite --namespace ns-ybzhw ` `kbcli cluster label gaussdb-bpsmmq --list --namespace ns-ybzhw ` NAME NAMESPACE LABELS gaussdb-bpsmmq ns-ybzhw app.kubernetes.io/instance=gaussdb-bpsmmq case.name=kbcli.test2 clusterdefinition.kubeblocks.io/name=gaussdb clusterversion.kubeblocks.io/name= label cluster case.name=kbcli.test2 Success `kbcli cluster label gaussdb-bpsmmq case.name- --namespace ns-ybzhw ` `kbcli cluster label gaussdb-bpsmmq --list --namespace ns-ybzhw ` NAME NAMESPACE LABELS gaussdb-bpsmmq ns-ybzhw app.kubernetes.io/instance=gaussdb-bpsmmq clusterdefinition.kubeblocks.io/name=gaussdb clusterversion.kubeblocks.io/name= delete cluster label case.name Success cluster connect `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"show max_connections;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-2 --namespace ns-ybzhw -- bash -c "su - omm" ` Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 03:28:45 UTC 2025 max_connections ----------------- 225 (1 row) connect cluster Success set max_connections to 225 insert batch data by db client Error from server (NotFound): pods "test-db-client-executionloop-gaussdb-bpsmmq" not found DB_CLIENT_BATCH_DATA_COUNT: `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-gaussdb-bpsmmq --namespace ns-ybzhw ` Error from server (NotFound): pods "test-db-client-executionloop-gaussdb-bpsmmq" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-executionloop-gaussdb-bpsmmq" not found `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres apiVersion: v1 kind: Pod metadata: name: test-db-client-executionloop-gaussdb-bpsmmq namespace: ns-ybzhw spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local" - "--user" - "dba" - "--password" - "511qH9i3MJ6g" - "--port" - "5432" - "--dbtype" - "gaussdb" - "--test" - "executionloop" - "--duration" - "60" - "--interval" - "1" restartPolicy: Never `kubectl apply -f test-db-client-executionloop-gaussdb-bpsmmq.yaml` pod/test-db-client-executionloop-gaussdb-bpsmmq created apply test-db-client-executionloop-gaussdb-bpsmmq.yaml Success `rm -rf test-db-client-executionloop-gaussdb-bpsmmq.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 1/1 Running 0 5s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 1/1 Running 0 10s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 1/1 Running 0 15s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 1/1 Running 0 20s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 1/1 Running 0 26s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 1/1 Running 0 31s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 1/1 Running 0 37s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 1/1 Running 0 42s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 1/1 Running 0 48s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 1/1 Running 0 53s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 1/1 Running 0 59s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 1/1 Running 0 64s check pod test-db-client-executionloop-gaussdb-bpsmmq status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-bpsmmq 0/1 Completed 0 70s check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Running Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-2;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-1 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-2 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done --host gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local --user dba --password 511qH9i3MJ6g --port 5432 --dbtype gaussdb --test executionloop --duration 60 --interval 1 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider] Sep 01, 2025 3:29:15 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [64643837-7458-4a6b-b9b2-78ca517d146c] Try to connect. IP: gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 Sep 01, 2025 3:29:16 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [10.244.1.228:33084/aussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local/10.0.110.236:5432] Connection is established. ID: 64643837-7458-4a6b-b9b2-78ca517d146c Sep 01, 2025 3:29:16 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: Connect complete. ID: 64643837-7458-4a6b-b9b2-78ca517d146c Execution loop start: create databases executions_loop CREATE DATABASE executions_loop; reconnect connection executions_loop Sep 01, 2025 3:29:18 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [574cec90-3365-49a4-a66a-3d439f2b2e35] Try to connect. IP: gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 Sep 01, 2025 3:29:18 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [10.244.1.228:33086/aussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local/10.0.110.236:5432] Connection is established. ID: 574cec90-3365-49a4-a66a-3d439f2b2e35 Sep 01, 2025 3:29:18 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: Connect complete. ID: 574cec90-3365-49a4-a66a-3d439f2b2e35 drop table executions_loop_table DROP TABLE IF EXISTS executions_loop_table; create table executions_loop_table CREATE TABLE IF NOT EXISTS executions_loop_table (id SERIAL PRIMARY KEY , value text); Execution loop start:INSERT INTO executions_loop_table (value) VALUES ('executions_loop_test_1'); [ 1s ] executions total: 1 successful: 1 failed: 0 disconnect: 0 [ 2s ] executions total: 102 successful: 102 failed: 0 disconnect: 0 [ 3s ] executions total: 250 successful: 250 failed: 0 disconnect: 0 [ 4s ] executions total: 341 successful: 341 failed: 0 disconnect: 0 [ 5s ] executions total: 479 successful: 479 failed: 0 disconnect: 0 [ 6s ] executions total: 583 successful: 583 failed: 0 disconnect: 0 [ 7s ] executions total: 724 successful: 724 failed: 0 disconnect: 0 [ 8s ] executions total: 812 successful: 812 failed: 0 disconnect: 0 [ 9s ] executions total: 928 successful: 928 failed: 0 disconnect: 0 [ 10s ] executions total: 1041 successful: 1041 failed: 0 disconnect: 0 [ 11s ] executions total: 1185 successful: 1185 failed: 0 disconnect: 0 [ 12s ] executions total: 1278 successful: 1278 failed: 0 disconnect: 0 [ 13s ] executions total: 1394 successful: 1394 failed: 0 disconnect: 0 [ 14s ] executions total: 1514 successful: 1514 failed: 0 disconnect: 0 [ 15s ] executions total: 1636 successful: 1636 failed: 0 disconnect: 0 [ 16s ] executions total: 1688 successful: 1688 failed: 0 disconnect: 0 [ 17s ] executions total: 1826 successful: 1826 failed: 0 disconnect: 0 [ 18s ] executions total: 1926 successful: 1926 failed: 0 disconnect: 0 [ 19s ] executions total: 2037 successful: 2037 failed: 0 disconnect: 0 [ 20s ] executions total: 2151 successful: 2151 failed: 0 disconnect: 0 [ 21s ] executions total: 2255 successful: 2255 failed: 0 disconnect: 0 [ 22s ] executions total: 2331 successful: 2331 failed: 0 disconnect: 0 [ 23s ] executions total: 2460 successful: 2460 failed: 0 disconnect: 0 [ 24s ] executions total: 2516 successful: 2516 failed: 0 disconnect: 0 [ 25s ] executions total: 2637 successful: 2637 failed: 0 disconnect: 0 [ 26s ] executions total: 2772 successful: 2772 failed: 0 disconnect: 0 [ 27s ] executions total: 2907 successful: 2907 failed: 0 disconnect: 0 [ 28s ] executions total: 3010 successful: 3010 failed: 0 disconnect: 0 [ 29s ] executions total: 3166 successful: 3166 failed: 0 disconnect: 0 [ 30s ] executions total: 3327 successful: 3327 failed: 0 disconnect: 0 [ 31s ] executions total: 3391 successful: 3391 failed: 0 disconnect: 0 [ 32s ] executions total: 3547 successful: 3547 failed: 0 disconnect: 0 [ 33s ] executions total: 3662 successful: 3662 failed: 0 disconnect: 0 [ 34s ] executions total: 3733 successful: 3733 failed: 0 disconnect: 0 [ 35s ] executions total: 3847 successful: 3847 failed: 0 disconnect: 0 [ 36s ] executions total: 3953 successful: 3953 failed: 0 disconnect: 0 [ 37s ] executions total: 4090 successful: 4090 failed: 0 disconnect: 0 [ 38s ] executions total: 4238 successful: 4238 failed: 0 disconnect: 0 [ 39s ] executions total: 4353 successful: 4353 failed: 0 disconnect: 0 [ 40s ] executions total: 4455 successful: 4455 failed: 0 disconnect: 0 [ 41s ] executions total: 4599 successful: 4599 failed: 0 disconnect: 0 [ 42s ] executions total: 4760 successful: 4760 failed: 0 disconnect: 0 [ 43s ] executions total: 4866 successful: 4866 failed: 0 disconnect: 0 [ 44s ] executions total: 4981 successful: 4981 failed: 0 disconnect: 0 [ 45s ] executions total: 5105 successful: 5105 failed: 0 disconnect: 0 [ 46s ] executions total: 5259 successful: 5259 failed: 0 disconnect: 0 [ 47s ] executions total: 5360 successful: 5360 failed: 0 disconnect: 0 [ 48s ] executions total: 5497 successful: 5497 failed: 0 disconnect: 0 [ 49s ] executions total: 5646 successful: 5646 failed: 0 disconnect: 0 [ 60s ] executions total: 5776 successful: 5776 failed: 0 disconnect: 0 Test Result: Total Executions: 5776 Successful Executions: 5776 Failed Executions: 0 Disconnection Counts: 0 Connection Information: Database Type: gaussdb Host: gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local Port: 5432 Database: Table: User: dba Org: Access Mode: mysql Test Type: executionloop Query: Duration: 60 seconds Interval: 1 seconds DB_CLIENT_BATCH_DATA_COUNT: 5776 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-gaussdb-bpsmmq --namespace ns-ybzhw ` pod/test-db-client-executionloop-gaussdb-bpsmmq patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-executionloop-gaussdb-bpsmmq" force deleted `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"DROP TABLE IF EXISTS tmp_table; CREATE TABLE IF NOT EXISTS tmp_table (id INT PRIMARY KEY , value text); INSERT INTO tmp_table (id,value) VALUES (1,'vzvtz');\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-2 --namespace ns-ybzhw -- bash -c "su - omm"` Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 03:30:30 UTC 2025 NOTICE: table "tmp_table" does not exist, skipping DROP TABLE NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "tmp_table_pkey" for table "tmp_table" CREATE TABLE INSERT 0 1 add consistent data vzvtz Success test failover fullcpu check cluster status before cluster-failover-fullcpu check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-gaussdb-bpsmmq" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-fullcpu-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-2 mode: all stressors: cpu: workers: 100 load: 100 duration: 2m `kubectl apply -f test-chaos-mesh-fullcpu-gaussdb-bpsmmq.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-fullcpu-gaussdb-bpsmmq created apply test-chaos-mesh-fullcpu-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-fullcpu-gaussdb-bpsmmq.yaml` fullcpu chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-2;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-1 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-2 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-gaussdb-bpsmmq" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-gaussdb-bpsmmq" not found check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-2 failover fullcpu Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-2 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 03:33:15 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-2 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover networkdelay check cluster status before cluster-failover-networkdelay check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-gaussdb-bpsmmq" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkdelay-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-2 mode: all action: delay delay: latency: 2000ms correlation: '100' jitter: 0ms direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkdelay-gaussdb-bpsmmq.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkdelay-gaussdb-bpsmmq created apply test-chaos-mesh-networkdelay-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-networkdelay-gaussdb-bpsmmq.yaml` networkdelay chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-2;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-1 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-2 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-gaussdb-bpsmmq" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-gaussdb-bpsmmq" not found check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-2 failover networkdelay Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-2 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 03:35:55 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-2 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover networkloss check cluster status before cluster-failover-networkloss check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkloss-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkloss-gaussdb-bpsmmq" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkloss-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkloss-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-2 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkloss-gaussdb-bpsmmq.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkloss-gaussdb-bpsmmq created apply test-chaos-mesh-networkloss-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-networkloss-gaussdb-bpsmmq.yaml` networkloss chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-0;secondary: gaussdb-bpsmmq-gaussdb-1 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkloss-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkloss-gaussdb-bpsmmq" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkloss-gaussdb-bpsmmq" not found check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-0 failover networkloss Failure `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 03:38:34 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover connectionstressover check cluster status before cluster-failover-connectionstressover check cluster status done cluster_status:Running check node drain check node drain success Error from server (NotFound): pods "test-db-client-connectionstressover-gaussdb-bpsmmq" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstressover-gaussdb-bpsmmq --namespace ns-ybzhw ` Error from server (NotFound): pods "test-db-client-connectionstressover-gaussdb-bpsmmq" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-connectionstressover-gaussdb-bpsmmq" not found `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres apiVersion: v1 kind: Pod metadata: name: test-db-client-connectionstressover-gaussdb-bpsmmq namespace: ns-ybzhw spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local" - "--user" - "dba" - "--password" - "511qH9i3MJ6g" - "--port" - "5432" - "--database" - "postgres" - "--dbtype" - "gaussdb" - "--test" - "connectionstress" - "--connections" - "225" - "--duration" - "60" restartPolicy: Never `kubectl apply -f test-db-client-connectionstressover-gaussdb-bpsmmq.yaml` pod/test-db-client-connectionstressover-gaussdb-bpsmmq created apply test-db-client-connectionstressover-gaussdb-bpsmmq.yaml Success `rm -rf test-db-client-connectionstressover-gaussdb-bpsmmq.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstressover-gaussdb-bpsmmq 1/1 Running 0 5s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstressover-gaussdb-bpsmmq 1/1 Running 0 10s check pod test-db-client-connectionstressover-gaussdb-bpsmmq status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstressover-gaussdb-bpsmmq 0/1 Completed 0 15s check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-1;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at com.huawei.opengauss.jdbc.core.PGStream.(PGStream.java:70) at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:119) at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:246) ... 11 more Sep 01, 2025 3:39:06 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [58f41b75-7e2e-4da4-8bdd-d95b8fb6bbee] Try to connect. IP: gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 Sep 01, 2025 3:39:06 AM org.opengauss.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [d02b90e7-16a6-4181-a833-0afb058345ab] Try to connect. IP: gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 Failed to connect to GaussDB database: com.huawei.opengauss.jdbc.util.PSQLException: Connection to gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. Trying with database GaussDB. Sep 01, 2025 3:39:06 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [0dfbedf4-f89f-43a7-8dbc-78ca65c8db67] Try to connect. IP: gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 Sep 01, 2025 3:39:06 AM org.opengauss.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [4f1df9ae-c72e-45ce-9240-22ab449b0509] Try to connect. IP: gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 java.io.IOException: Failed to connect to GaussDB database: at com.apecloud.dbtester.tester.GaussdbTester.connect(GaussdbTester.java:64) at com.apecloud.dbtester.tester.GaussdbTester.connectionStress(GaussdbTester.java:113) at com.apecloud.dbtester.commons.TestExecutor.executeTest(TestExecutor.java:33) at OneClient.executeTest(OneClient.java:108) at OneClient.main(OneClient.java:40) Caused by: com.huawei.opengauss.jdbc.util.PSQLException: Connection to gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:370) at com.huawei.opengauss.jdbc.core.ConnectionFactory.openConnection(ConnectionFactory.java:57) at com.huawei.opengauss.jdbc.jdbc.PgConnection.(PgConnection.java:274) at com.huawei.opengauss.jdbc.Driver.makeConnection(Driver.java:576) at com.huawei.opengauss.jdbc.Driver.connect(Driver.java:315) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.GaussdbTester.connect(GaussdbTester.java:62) ... 4 more Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at com.huawei.opengauss.jdbc.core.PGStream.(PGStream.java:70) at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:119) at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:246) ... 11 more Sep 01, 2025 3:39:06 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [5fb6a3fc-0b6c-4b91-ac54-048cad9539da] Try to connect. IP: gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 Sep 01, 2025 3:39:06 AM org.opengauss.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [b426718a-d34c-4633-a99c-97f63a4b5dec] Try to connect. IP: gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 Failed to connect to GaussDB database: com.huawei.opengauss.jdbc.util.PSQLException: Connection to gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. Trying with database GaussDB. Sep 01, 2025 3:39:06 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [038dce22-ceba-4c2e-9326-a019a7245323] Try to connect. IP: gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 Sep 01, 2025 3:39:06 AM org.opengauss.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [f1edb593-1447-4cbf-87c1-569c2c2d1a6e] Try to connect. IP: gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 java.io.IOException: Failed to connect to GaussDB database: at com.apecloud.dbtester.tester.GaussdbTester.connect(GaussdbTester.java:64) at com.apecloud.dbtester.tester.GaussdbTester.connectionStress(GaussdbTester.java:113) at com.apecloud.dbtester.commons.TestExecutor.executeTest(TestExecutor.java:33) at OneClient.executeTest(OneClient.java:108) at OneClient.main(OneClient.java:40) Caused by: com.huawei.opengauss.jdbc.util.PSQLException: Connection to gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:370) at com.huawei.opengauss.jdbc.core.ConnectionFactory.openConnection(ConnectionFactory.java:57) at com.huawei.opengauss.jdbc.jdbc.PgConnection.(PgConnection.java:274) at com.huawei.opengauss.jdbc.Driver.makeConnection(Driver.java:576) at com.huawei.opengauss.jdbc.Driver.connect(Driver.java:315) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.GaussdbTester.connect(GaussdbTester.java:62) ... 4 more Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at com.huawei.opengauss.jdbc.core.PGStream.(PGStream.java:70) at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:119) at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:246) ... 11 more Test Result: null Connection Information: Database Type: gaussdb Host: gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local Port: 5432 Database: postgres Table: User: dba Org: Access Mode: mysql Test Type: connectionstress Connection Count: 225 Duration: 60 seconds `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstressover-gaussdb-bpsmmq --namespace ns-ybzhw ` pod/test-db-client-connectionstressover-gaussdb-bpsmmq patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-connectionstressover-gaussdb-bpsmmq" force deleted check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-1 failover connectionstressover Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 03:40:00 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover networkcorrupt check cluster status before cluster-failover-networkcorrupt check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorrupt-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorrupt-gaussdb-bpsmmq" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorrupt-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorrupt-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-1 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorrupt-gaussdb-bpsmmq.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorrupt-gaussdb-bpsmmq created apply test-chaos-mesh-networkcorrupt-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-networkcorrupt-gaussdb-bpsmmq.yaml` networkcorrupt chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:24 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-0;secondary: gaussdb-bpsmmq-gaussdb-1 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorrupt-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorrupt-gaussdb-bpsmmq" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorrupt-gaussdb-bpsmmq" not found check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-0 failover networkcorrupt Failure `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 03:42:46 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover networkpartition check cluster status before cluster-failover-networkpartition check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-gaussdb-bpsmmq" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkpartition-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-0 action: partition mode: all target: mode: all selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-1 direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkpartition-gaussdb-bpsmmq.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkpartition-gaussdb-bpsmmq created apply test-chaos-mesh-networkpartition-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-networkpartition-gaussdb-bpsmmq.yaml` networkpartition chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "gaussdb-bpsmmq-gaussdb-1" force deleted Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "gaussdb-bpsmmq-gaussdb-2" force deleted cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 11:44 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:47 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 11:47 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-0;secondary: gaussdb-bpsmmq-gaussdb-1 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-gaussdb-bpsmmq" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-gaussdb-bpsmmq" not found check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-0 failover networkpartition Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 03:50:09 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart gaussdb-bpsmmq --auto-approve --force=true --namespace ns-ybzhw ` OpsRequest gaussdb-bpsmmq-restart-c9dh9 created successfully, you can view the progress: kbcli cluster describe-ops gaussdb-bpsmmq-restart-c9dh9 -n ns-ybzhw check ops status `kbcli cluster list-ops gaussdb-bpsmmq --status all --namespace ns-ybzhw ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-bpsmmq-restart-c9dh9 ns-ybzhw Restart gaussdb-bpsmmq gaussdb Running 0/3 Sep 01,2025 11:50 UTC+0800 check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating [Error] check cluster status timeout --------------------------------------get cluster gaussdb-bpsmmq yaml-------------------------------------- `kubectl get cluster gaussdb-bpsmmq -o yaml --namespace ns-ybzhw ` apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: annotations: kubeblocks.io/ops-request: '[***"name":"gaussdb-bpsmmq-restart-c9dh9","type":"Restart"***]' kubeblocks.io/reconcile: "2025-09-01T03:55:40.856449852Z" kubectl.kubernetes.io/last-applied-configuration: | ***"apiVersion":"apps.kubeblocks.io/v1alpha1","kind":"Cluster","metadata":***"annotations":***,"name":"gaussdb-bpsmmq","namespace":"ns-ybzhw"***,"spec":***"clusterDefinitionRef":"gaussdb","componentSpecs":[***"name":"gaussdb","replicas":3,"resources":***"limits":***"cpu":"500m","memory":"2Gi"***,"requests":***"cpu":"500m","memory":"2Gi"***,"serviceVersion":"2.23.1","volumeClaimTemplates":[***"name":"data","spec":***"accessModes":["ReadWriteOnce"],"resources":***"requests":***"storage":"20Gi"***,"storageClassName":null***]***],"terminationPolicy":"WipeOut","topology":"replication"*** creationTimestamp: "2025-09-01T03:24:00Z" finalizers: - cluster.kubeblocks.io/finalizer generation: 3 labels: app.kubernetes.io/instance: gaussdb-bpsmmq clusterdefinition.kubeblocks.io/name: gaussdb clusterversion.kubeblocks.io/name: "" name: gaussdb-bpsmmq namespace: ns-ybzhw resourceVersion: "48931" uid: bb2e7a02-44d9-4812-b6ed-80ad74100627 spec: clusterDefinitionRef: gaussdb componentSpecs: - componentDef: gaussdb-repl name: gaussdb replicas: 3 resources: limits: cpu: 500m memory: 2Gi requests: cpu: 500m memory: 2Gi serviceVersion: 2.23.1 volumeClaimTemplates: - name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi resources: cpu: "0" memory: "0" storage: size: "0" terminationPolicy: WipeOut topology: replication status: clusterDefGeneration: 1 components: gaussdb: message: InstanceSet/gaussdb-bpsmmq-gaussdb: '["gaussdb-bpsmmq-gaussdb-0"]' phase: Updating podsReady: false podsReadyTime: "2025-09-01T03:50:01Z" conditions: - lastTransitionTime: "2025-09-01T03:24:00Z" message: 'The operator has started the provisioning of Cluster: gaussdb-bpsmmq' observedGeneration: 3 reason: PreCheckSucceed status: "True" type: ProvisioningStarted - lastTransitionTime: "2025-09-01T03:24:00Z" message: Successfully applied for resources observedGeneration: 3 reason: ApplyResourcesSucceed status: "True" type: ApplyResources - lastTransitionTime: "2025-09-01T03:50:23Z" message: 'pods are not ready in Components: [gaussdb], refer to related component message in Cluster.status.components' reason: ReplicasNotReady status: "False" type: ReplicasReady - lastTransitionTime: "2025-09-01T03:50:23Z" message: 'pods are unavailable in Components: [gaussdb], refer to related component message in Cluster.status.components' reason: ComponentsNotReady status: "False" type: Ready observedGeneration: 3 phase: Updating ------------------------------------------------------------------------------------------------------------------ --------------------------------------describe cluster gaussdb-bpsmmq-------------------------------------- `kubectl describe cluster gaussdb-bpsmmq --namespace ns-ybzhw ` Name: gaussdb-bpsmmq Namespace: ns-ybzhw Labels: app.kubernetes.io/instance=gaussdb-bpsmmq clusterdefinition.kubeblocks.io/name=gaussdb clusterversion.kubeblocks.io/name= Annotations: kubeblocks.io/ops-request: [***"name":"gaussdb-bpsmmq-restart-c9dh9","type":"Restart"***] kubeblocks.io/reconcile: 2025-09-01T03:55:40.856449852Z API Version: apps.kubeblocks.io/v1alpha1 Kind: Cluster Metadata: Creation Timestamp: 2025-09-01T03:24:00Z Finalizers: cluster.kubeblocks.io/finalizer Generation: 3 Resource Version: 48931 UID: bb2e7a02-44d9-4812-b6ed-80ad74100627 Spec: Cluster Definition Ref: gaussdb Component Specs: Component Def: gaussdb-repl Name: gaussdb Replicas: 3 Resources: Limits: Cpu: 500m Memory: 2Gi Requests: Cpu: 500m Memory: 2Gi Service Version: 2.23.1 Volume Claim Templates: Name: data Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 20Gi Resources: Cpu: 0 Memory: 0 Storage: Size: 0 Termination Policy: WipeOut Topology: replication Status: Cluster Def Generation: 1 Components: Gaussdb: Message: InstanceSet/gaussdb-bpsmmq-gaussdb: ["gaussdb-bpsmmq-gaussdb-0"] Phase: Updating Pods Ready: false Pods Ready Time: 2025-09-01T03:50:01Z Conditions: Last Transition Time: 2025-09-01T03:24:00Z Message: The operator has started the provisioning of Cluster: gaussdb-bpsmmq Observed Generation: 3 Reason: PreCheckSucceed Status: True Type: ProvisioningStarted Last Transition Time: 2025-09-01T03:24:00Z Message: Successfully applied for resources Observed Generation: 3 Reason: ApplyResourcesSucceed Status: True Type: ApplyResources Last Transition Time: 2025-09-01T03:50:23Z Message: pods are not ready in Components: [gaussdb], refer to related component message in Cluster.status.components Reason: ReplicasNotReady Status: False Type: ReplicasReady Last Transition Time: 2025-09-01T03:50:23Z Message: pods are unavailable in Components: [gaussdb], refer to related component message in Cluster.status.components Reason: ComponentsNotReady Status: False Type: Ready Observed Generation: 3 Phase: Updating Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal PreCheckSucceed 34m (x2 over 34m) cluster-controller The operator has started the provisioning of Cluster: gaussdb-bpsmmq Normal ApplyResourcesSucceed 34m (x2 over 34m) cluster-controller Successfully applied for resources Normal ComponentPhaseTransition 34m cluster-controller component is Creating Warning Unhealthy 30m event-controller Pod gaussdb-bpsmmq-gaussdb-0: Readiness probe failed: gsql: could not connect to server: Operation now in progress Is the server running on host "127.0.0.1" and accepting TCP/IP connections on port 5432? Warning Unhealthy 25m (x3 over 26m) event-controller Pod gaussdb-bpsmmq-gaussdb-2: Readiness probe failed: command timed out: "bash -c LD_LIBRARY_PATH=$GAUSSHOME/lib:$LD_LIBRARY_PATH gsql -h 127.0.0.1 -p \"$***GAUSSDB_BASEPORT***\" -U $***GAUSSDB_USER*** -W \"$***GAUSSDB_PASSWORD***\" -dpostgres -c \"select 1\"\n" timed out after 5s Normal ClusterReady 22m (x3 over 29m) cluster-controller Cluster: gaussdb-bpsmmq is ready, current phase is Running Normal AllReplicasReady 22m (x3 over 29m) cluster-controller all pods of components are ready, waiting for the probe detection successful Normal Running 22m (x3 over 29m) cluster-controller Cluster: gaussdb-bpsmmq is ready, current phase is Running Warning ReplicasNotReady 21m (x3 over 26m) cluster-controller pods are not ready in Components: [gaussdb], refer to related component message in Cluster.status.components Warning ComponentsNotReady 21m (x3 over 26m) cluster-controller pods are unavailable in Components: [gaussdb], refer to related component message in Cluster.status.components Warning Unhealthy 21m (x12 over 26m) event-controller Pod gaussdb-bpsmmq-gaussdb-2: Readiness probe failed: Get "http://10.244.5.193:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Normal ComponentPhaseTransition 19m (x4 over 29m) cluster-controller component is Running Warning Unhealthy 17m event-controller Pod gaussdb-bpsmmq-gaussdb-0: Readiness probe failed: Get "http://10.244.5.185:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Warning Unhealthy 16m (x10 over 17m) event-controller Pod gaussdb-bpsmmq-gaussdb-1: Readiness probe failed: Get "http://10.244.5.73:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Normal ComponentPhaseTransition 14m (x6 over 26m) cluster-controller component is Updating Warning Unhealthy 13m event-controller Pod gaussdb-bpsmmq-gaussdb-0: Readiness probe failed: Get "http://10.244.5.185:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Warning Unhealthy 13m event-controller Pod gaussdb-bpsmmq-gaussdb-1: Readiness probe failed: gsql: FATAL: can not accept connection in pending mode. Warning Unhealthy 13m event-controller Pod gaussdb-bpsmmq-gaussdb-2: Readiness probe failed: Get "http://10.244.5.193:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Warning Unhealthy 13m event-controller Pod gaussdb-bpsmmq-gaussdb-1: Readiness probe failed: Get "http://10.244.5.73:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Warning Unhealthy 13m event-controller Pod gaussdb-bpsmmq-gaussdb-2: Readiness probe failed: command timed out: "bash -c LD_LIBRARY_PATH=$GAUSSHOME/lib:$LD_LIBRARY_PATH gsql -h 127.0.0.1 -p \"$***GAUSSDB_BASEPORT***\" -U $***GAUSSDB_USER*** -W \"$***GAUSSDB_PASSWORD***\" -dpostgres -c \"select 1\"\n" timed out after 5s Normal ComponentPhaseTransition 11m cluster-controller component is Failed Warning Failed 11m cluster-controller Cluster: gaussdb-bpsmmq is Failed, check according to the components message Warning Unhealthy 10m (x3 over 13m) event-controller Pod gaussdb-bpsmmq-gaussdb-0: Readiness probe failed: gsql: could not connect to server: Operation now in progress Is the server running on host "127.0.0.1" and accepting TCP/IP connections on port 5432? Warning Unhealthy 8m50s (x3 over 9m21s) event-controller Pod gaussdb-bpsmmq-gaussdb-0: Readiness probe failed: gsql: FATAL: can not accept connection in pending mode. Normal ComponentPhaseTransition 8m14s cluster-controller component is Running Normal AllReplicasReady 8m14s cluster-controller all pods of components are ready, waiting for the probe detection successful Normal ClusterReady 8m14s cluster-controller Cluster: gaussdb-bpsmmq is ready, current phase is Running Normal Running 8m14s cluster-controller Cluster: gaussdb-bpsmmq is ready, current phase is Running Normal ComponentPhaseTransition 7m52s (x2 over 11m) cluster-controller component is Updating Normal PreCheckSucceed 7m52s cluster-controller The operator has started the provisioning of Cluster: gaussdb-bpsmmq Normal ApplyResourcesSucceed 7m52s cluster-controller Successfully applied for resources Warning ReplicasNotReady 7m52s cluster-controller pods are not ready in Components: [gaussdb], refer to related component message in Cluster.status.components Warning ComponentsNotReady 7m52s cluster-controller pods are unavailable in Components: [gaussdb], refer to related component message in Cluster.status.components Warning FailedScheduling 2m35s event-controller Pod gaussdb-bpsmmq-gaussdb-1: 0/3 nodes are available: 1 Insufficient memory, 1 node(s) exceed max volume count, 1 node(s) were unschedulable. preemption: 0/3 nodes are available: 1 Preemption is not helpful for scheduling, 2 No preemption victims found for incoming pod. Warning FailedScheduling 2m35s event-controller Pod gaussdb-bpsmmq-gaussdb-2: 0/3 nodes are available: 1 Insufficient memory, 1 node(s) exceed max volume count, 1 node(s) were unschedulable. preemption: 0/3 nodes are available: 1 Preemption is not helpful for scheduling, 2 No preemption victims found for incoming pod. ------------------------------------------------------------------------------------------------------------------ check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 11:54 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000007/10.224.0.9 Sep 01,2025 11:55 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000007/10.224.0.9 Sep 01,2025 11:50 UTC+0800 check pod status done check cluster status again check ops status `kbcli cluster list-ops gaussdb-bpsmmq --status all --namespace ns-ybzhw ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-bpsmmq-restart-c9dh9 ns-ybzhw Restart gaussdb-bpsmmq gaussdb Running 2/3 Sep 01,2025 11:50 UTC+0800 ops_status:gaussdb-bpsmmq-restart-c9dh9 ns-ybzhw Restart gaussdb-bpsmmq gaussdb Running 2/3 Sep 01,2025 11:50 UTC+0800 ops_status:gaussdb-bpsmmq-restart-c9dh9 ns-ybzhw Restart gaussdb-bpsmmq gaussdb Running 2/3 Sep 01,2025 11:50 UTC+0800 ops_status:gaussdb-bpsmmq-restart-c9dh9 ns-ybzhw Restart gaussdb-bpsmmq gaussdb Running 2/3 Sep 01,2025 11:50 UTC+0800 ops_status:gaussdb-bpsmmq-restart-c9dh9 ns-ybzhw Restart gaussdb-bpsmmq gaussdb Running 2/3 Sep 01,2025 11:50 UTC+0800 ops_status:gaussdb-bpsmmq-restart-c9dh9 ns-ybzhw Restart gaussdb-bpsmmq gaussdb Running 2/3 Sep 01,2025 11:50 UTC+0800 ops_status:gaussdb-bpsmmq-restart-c9dh9 ns-ybzhw Restart gaussdb-bpsmmq gaussdb Running 2/3 Sep 01,2025 11:50 UTC+0800 ops_status:gaussdb-bpsmmq-restart-c9dh9 ns-ybzhw Restart gaussdb-bpsmmq gaussdb Running 3/3 Sep 01,2025 11:50 UTC+0800 ops_status:gaussdb-bpsmmq-restart-c9dh9 ns-ybzhw Restart gaussdb-bpsmmq gaussdb Running 3/3 Sep 01,2025 11:50 UTC+0800 check ops status done ops_status:gaussdb-bpsmmq-restart-c9dh9 ns-ybzhw Restart gaussdb-bpsmmq gaussdb Succeed 3/3 Sep 01,2025 11:50 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests gaussdb-bpsmmq-restart-c9dh9 --namespace ns-ybzhw ` opsrequest.apps.kubeblocks.io/gaussdb-bpsmmq-restart-c9dh9 patched `kbcli cluster delete-ops --name gaussdb-bpsmmq-restart-c9dh9 --force --auto-approve --namespace ns-ybzhw ` OpsRequest gaussdb-bpsmmq-restart-c9dh9 deleted `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 03:54:21 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover networkbandwidth check cluster status before cluster-failover-networkbandwidth check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidth-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidth-gaussdb-bpsmmq" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidth-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkbandwidth-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-0 action: bandwidth mode: all bandwidth: rate: '1bps' limit: 20971520 buffer: 10000 duration: 2m `kubectl apply -f test-chaos-mesh-networkbandwidth-gaussdb-bpsmmq.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkbandwidth-gaussdb-bpsmmq created apply test-chaos-mesh-networkbandwidth-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-networkbandwidth-gaussdb-bpsmmq.yaml` networkbandwidth chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 11:54 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:01 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:01 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-1;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidth-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidth-gaussdb-bpsmmq" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidth-gaussdb-bpsmmq" not found check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-1 failover networkbandwidth Failure `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:03:42 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success cluster update monitor cluster does not need to check monitor currently check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Running Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 11:54 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:01 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:01 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-1;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done test failover dnsrandom check cluster status before cluster-failover-dnsrandom check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-gaussdb-bpsmmq" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnsrandom-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-1 mode: all action: random duration: 2m `kubectl apply -f test-chaos-mesh-dnsrandom-gaussdb-bpsmmq.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnsrandom-gaussdb-bpsmmq created apply test-chaos-mesh-dnsrandom-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-dnsrandom-gaussdb-bpsmmq.yaml` dnsrandom chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Running Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 11:54 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:01 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:01 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-1;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-gaussdb-bpsmmq" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-gaussdb-bpsmmq" not found check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-1 failover dnsrandom Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:06:21 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover oom check cluster status before cluster-failover-oom check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-gaussdb-bpsmmq" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-1 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-gaussdb-bpsmmq.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-gaussdb-bpsmmq created apply test-chaos-mesh-oom-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-oom-gaussdb-bpsmmq.yaml` check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 11:54 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:01 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:01 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-0;secondary: gaussdb-bpsmmq-gaussdb-1 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-gaussdb-bpsmmq" force deleted stresschaos.chaos-mesh.org/test-chaos-mesh-oom-gaussdb-bpsmmq patched check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-0 failover oom Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:07:32 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover podkill check cluster status before cluster-failover-podkill check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podkill-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podkill-gaussdb-bpsmmq" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podkill-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podkill-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-0 mode: all action: pod-kill `kubectl apply -f test-chaos-mesh-podkill-gaussdb-bpsmmq.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podkill-gaussdb-bpsmmq created apply test-chaos-mesh-podkill-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-podkill-gaussdb-bpsmmq.yaml` check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:07 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:01 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:01 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-1;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podkill-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podkill-gaussdb-bpsmmq" force deleted podchaos.chaos-mesh.org/test-chaos-mesh-podkill-gaussdb-bpsmmq patched check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-1 failover podkill Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:08:54 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover drainnode check cluster status before cluster-failover-drainnode check cluster status done cluster_status:Running check node drain check node drain success kubectl get pod gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -o jsonpath='***.spec.nodeName***' get node name:aks-cicdamdpool-38368475-vmss000006 success check if multiple pods are on the same node kubectl get pod gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -o jsonpath='***.spec.nodeName***' get node name:aks-cicdamdpool-38368475-vmss000006 success Multiple pods on the same node test failover timeoffset check cluster status before cluster-failover-timeoffset check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-gaussdb-bpsmmq" not found Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: TimeChaos metadata: name: test-chaos-mesh-timeoffset-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-1 mode: all timeOffset: '-10m' clockIds: - CLOCK_REALTIME duration: 2m `kubectl apply -f test-chaos-mesh-timeoffset-gaussdb-bpsmmq.yaml` timechaos.chaos-mesh.org/test-chaos-mesh-timeoffset-gaussdb-bpsmmq created apply test-chaos-mesh-timeoffset-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-timeoffset-gaussdb-bpsmmq.yaml` timeoffset chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Running Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:07 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:01 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:01 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-1;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-gaussdb-bpsmmq" force deleted Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-gaussdb-bpsmmq" not found check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-1 failover timeoffset Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:11:23 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover networkduplicate check cluster status before cluster-failover-networkduplicate check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-gaussdb-bpsmmq" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkduplicate-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-1 mode: all action: duplicate duplicate: duplicate: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkduplicate-gaussdb-bpsmmq.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkduplicate-gaussdb-bpsmmq created apply test-chaos-mesh-networkduplicate-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-networkduplicate-gaussdb-bpsmmq.yaml` networkduplicate chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Running Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:07 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:01 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:01 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-1;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-gaussdb-bpsmmq" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-gaussdb-bpsmmq" not found check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-1 failover networkduplicate Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:13:47 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover dnserror check cluster status before cluster-failover-dnserror check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-gaussdb-bpsmmq" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnserror-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-1 mode: all action: error duration: 2m `kubectl apply -f test-chaos-mesh-dnserror-gaussdb-bpsmmq.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnserror-gaussdb-bpsmmq created apply test-chaos-mesh-dnserror-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-dnserror-gaussdb-bpsmmq.yaml` dnserror chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Running Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:07 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:01 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:01 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-1;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-gaussdb-bpsmmq" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-gaussdb-bpsmmq" not found check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-1 failover dnserror Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:16:12 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover kill1 check cluster status before cluster-failover-kill1 check cluster status done cluster_status:Running check node drain check node drain success `kill 1` Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file exec return message: check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Running Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:07 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:01 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:01 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-1;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-1 failover kill1 Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:16:46 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success cluster configure component_tmp: gaussdb check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster configure gaussdb-bpsmmq --auto-approve --force=true --set alarm_report_interval=20 --components gaussdb --config-spec gaussdb-config --config-file postgresql.conf --namespace ns-ybzhw ` Will updated configure file meta: ConfigSpec: gaussdb-config ConfigFile: postgresql.conf ComponentName: gaussdb ClusterName: gaussdb-bpsmmq OpsRequest gaussdb-bpsmmq-reconfiguring-mnvcj created successfully, you can view the progress: kbcli cluster describe-ops gaussdb-bpsmmq-reconfiguring-mnvcj -n ns-ybzhw check ops status `kbcli cluster list-ops gaussdb-bpsmmq --status all --namespace ns-ybzhw ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-bpsmmq-reconfiguring-mnvcj ns-ybzhw Reconfiguring gaussdb-bpsmmq gaussdb Running -/- Sep 01,2025 12:17 UTC+0800 check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Running Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:07 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:01 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 500m / 500m 2Gi / 2Gi data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:01 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-1;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done check ops status `kbcli cluster list-ops gaussdb-bpsmmq --status all --namespace ns-ybzhw ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-bpsmmq-reconfiguring-mnvcj ns-ybzhw Reconfiguring gaussdb-bpsmmq gaussdb Succeed 3/3 Sep 01,2025 12:17 UTC+0800 check ops status done ops_status:gaussdb-bpsmmq-reconfiguring-mnvcj ns-ybzhw Reconfiguring gaussdb-bpsmmq gaussdb Succeed 3/3 Sep 01,2025 12:17 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests gaussdb-bpsmmq-reconfiguring-mnvcj --namespace ns-ybzhw ` opsrequest.apps.kubeblocks.io/gaussdb-bpsmmq-reconfiguring-mnvcj patched `kbcli cluster delete-ops --name gaussdb-bpsmmq-reconfiguring-mnvcj --force --auto-approve --namespace ns-ybzhw ` OpsRequest gaussdb-bpsmmq-reconfiguring-mnvcj deleted check config variables Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file current value_actual: 20 configure:[alarm_report_interval] result actual:[20] equal expected:[20] `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:17:25 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success cluster vscale check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale gaussdb-bpsmmq --auto-approve --force=true --components gaussdb --cpu 600m --memory 2.1Gi --namespace ns-ybzhw ` OpsRequest gaussdb-bpsmmq-verticalscaling-rnbch created successfully, you can view the progress: kbcli cluster describe-ops gaussdb-bpsmmq-verticalscaling-rnbch -n ns-ybzhw check ops status `kbcli cluster list-ops gaussdb-bpsmmq --status all --namespace ns-ybzhw ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-bpsmmq-verticalscaling-rnbch ns-ybzhw VerticalScaling gaussdb-bpsmmq gaussdb Running 0/3 Sep 01,2025 12:17 UTC+0800 check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating [Error] check cluster status timeout --------------------------------------get cluster gaussdb-bpsmmq yaml-------------------------------------- `kubectl get cluster gaussdb-bpsmmq -o yaml --namespace ns-ybzhw ` apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: annotations: kubeblocks.io/ops-request: '[***"name":"gaussdb-bpsmmq-verticalscaling-rnbch","type":"VerticalScaling"***]' kubeblocks.io/reconcile: "2025-09-01T04:01:14.083314304Z" kubectl.kubernetes.io/last-applied-configuration: | ***"apiVersion":"apps.kubeblocks.io/v1alpha1","kind":"Cluster","metadata":***"annotations":***,"name":"gaussdb-bpsmmq","namespace":"ns-ybzhw"***,"spec":***"clusterDefinitionRef":"gaussdb","componentSpecs":[***"name":"gaussdb","replicas":3,"resources":***"limits":***"cpu":"500m","memory":"2Gi"***,"requests":***"cpu":"500m","memory":"2Gi"***,"serviceVersion":"2.23.1","volumeClaimTemplates":[***"name":"data","spec":***"accessModes":["ReadWriteOnce"],"resources":***"requests":***"storage":"20Gi"***,"storageClassName":null***]***],"terminationPolicy":"WipeOut","topology":"replication"*** creationTimestamp: "2025-09-01T03:24:00Z" finalizers: - cluster.kubeblocks.io/finalizer generation: 4 labels: app.kubernetes.io/instance: gaussdb-bpsmmq clusterdefinition.kubeblocks.io/name: gaussdb clusterversion.kubeblocks.io/name: "" name: gaussdb-bpsmmq namespace: ns-ybzhw resourceVersion: "69688" uid: bb2e7a02-44d9-4812-b6ed-80ad74100627 spec: clusterDefinitionRef: gaussdb componentSpecs: - componentDef: gaussdb-repl name: gaussdb replicas: 3 resources: limits: cpu: 600m memory: 2254857830400m requests: cpu: 600m memory: 2254857830400m serviceVersion: 2.23.1 volumeClaimTemplates: - name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi resources: cpu: "0" memory: "0" storage: size: "0" terminationPolicy: WipeOut topology: replication status: clusterDefGeneration: 1 components: gaussdb: message: InstanceSet/gaussdb-bpsmmq-gaussdb: '["gaussdb-bpsmmq-gaussdb-0"]' phase: Updating podsReady: false podsReadyTime: "2025-09-01T04:17:02Z" conditions: - lastTransitionTime: "2025-09-01T03:24:00Z" message: 'The operator has started the provisioning of Cluster: gaussdb-bpsmmq' observedGeneration: 4 reason: PreCheckSucceed status: "True" type: ProvisioningStarted - lastTransitionTime: "2025-09-01T03:24:00Z" message: Successfully applied for resources observedGeneration: 4 reason: ApplyResourcesSucceed status: "True" type: ApplyResources - lastTransitionTime: "2025-09-01T04:17:38Z" message: 'pods are not ready in Components: [gaussdb], refer to related component message in Cluster.status.components' reason: ReplicasNotReady status: "False" type: ReplicasReady - lastTransitionTime: "2025-09-01T04:17:38Z" message: 'pods are unavailable in Components: [gaussdb], refer to related component message in Cluster.status.components' reason: ComponentsNotReady status: "False" type: Ready observedGeneration: 4 phase: Updating ------------------------------------------------------------------------------------------------------------------ --------------------------------------describe cluster gaussdb-bpsmmq-------------------------------------- `kubectl describe cluster gaussdb-bpsmmq --namespace ns-ybzhw ` Name: gaussdb-bpsmmq Namespace: ns-ybzhw Labels: app.kubernetes.io/instance=gaussdb-bpsmmq clusterdefinition.kubeblocks.io/name=gaussdb clusterversion.kubeblocks.io/name= Annotations: kubeblocks.io/ops-request: [***"name":"gaussdb-bpsmmq-verticalscaling-rnbch","type":"VerticalScaling"***] kubeblocks.io/reconcile: 2025-09-01T04:01:14.083314304Z API Version: apps.kubeblocks.io/v1alpha1 Kind: Cluster Metadata: Creation Timestamp: 2025-09-01T03:24:00Z Finalizers: cluster.kubeblocks.io/finalizer Generation: 4 Resource Version: 69688 UID: bb2e7a02-44d9-4812-b6ed-80ad74100627 Spec: Cluster Definition Ref: gaussdb Component Specs: Component Def: gaussdb-repl Name: gaussdb Replicas: 3 Resources: Limits: Cpu: 600m Memory: 2254857830400m Requests: Cpu: 600m Memory: 2254857830400m Service Version: 2.23.1 Volume Claim Templates: Name: data Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 20Gi Resources: Cpu: 0 Memory: 0 Storage: Size: 0 Termination Policy: WipeOut Topology: replication Status: Cluster Def Generation: 1 Components: Gaussdb: Message: InstanceSet/gaussdb-bpsmmq-gaussdb: ["gaussdb-bpsmmq-gaussdb-0"] Phase: Updating Pods Ready: false Pods Ready Time: 2025-09-01T04:17:02Z Conditions: Last Transition Time: 2025-09-01T03:24:00Z Message: The operator has started the provisioning of Cluster: gaussdb-bpsmmq Observed Generation: 4 Reason: PreCheckSucceed Status: True Type: ProvisioningStarted Last Transition Time: 2025-09-01T03:24:00Z Message: Successfully applied for resources Observed Generation: 4 Reason: ApplyResourcesSucceed Status: True Type: ApplyResources Last Transition Time: 2025-09-01T04:17:38Z Message: pods are not ready in Components: [gaussdb], refer to related component message in Cluster.status.components Reason: ReplicasNotReady Status: False Type: ReplicasReady Last Transition Time: 2025-09-01T04:17:38Z Message: pods are unavailable in Components: [gaussdb], refer to related component message in Cluster.status.components Reason: ComponentsNotReady Status: False Type: Ready Observed Generation: 4 Phase: Updating Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Unhealthy 57m event-controller Pod gaussdb-bpsmmq-gaussdb-0: Readiness probe failed: gsql: could not connect to server: Operation now in progress Is the server running on host "127.0.0.1" and accepting TCP/IP connections on port 5432? Warning Unhealthy 52m (x3 over 53m) event-controller Pod gaussdb-bpsmmq-gaussdb-2: Readiness probe failed: command timed out: "bash -c LD_LIBRARY_PATH=$GAUSSHOME/lib:$LD_LIBRARY_PATH gsql -h 127.0.0.1 -p \"$***GAUSSDB_BASEPORT***\" -U $***GAUSSDB_USER*** -W \"$***GAUSSDB_PASSWORD***\" -dpostgres -c \"select 1\"\n" timed out after 5s Normal AllReplicasReady 49m (x3 over 57m) cluster-controller all pods of components are ready, waiting for the probe detection successful Normal ClusterReady 49m (x3 over 57m) cluster-controller Cluster: gaussdb-bpsmmq is ready, current phase is Running Normal Running 49m (x3 over 57m) cluster-controller Cluster: gaussdb-bpsmmq is ready, current phase is Running Warning ComponentsNotReady 49m (x3 over 54m) cluster-controller pods are unavailable in Components: [gaussdb], refer to related component message in Cluster.status.components Warning ReplicasNotReady 49m (x3 over 54m) cluster-controller pods are not ready in Components: [gaussdb], refer to related component message in Cluster.status.components Warning Unhealthy 48m (x12 over 53m) event-controller Pod gaussdb-bpsmmq-gaussdb-2: Readiness probe failed: Get "http://10.244.5.193:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Normal ComponentPhaseTransition 47m (x4 over 57m) cluster-controller component is Running Warning Unhealthy 44m event-controller Pod gaussdb-bpsmmq-gaussdb-0: Readiness probe failed: Get "http://10.244.5.185:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Warning Unhealthy 43m (x10 over 44m) event-controller Pod gaussdb-bpsmmq-gaussdb-1: Readiness probe failed: Get "http://10.244.5.73:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Normal ComponentPhaseTransition 41m (x6 over 54m) cluster-controller component is Updating Warning Unhealthy 40m event-controller Pod gaussdb-bpsmmq-gaussdb-1: Readiness probe failed: gsql: FATAL: can not accept connection in pending mode. Warning Unhealthy 40m event-controller Pod gaussdb-bpsmmq-gaussdb-0: Readiness probe failed: Get "http://10.244.5.185:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Warning Unhealthy 40m event-controller Pod gaussdb-bpsmmq-gaussdb-2: Readiness probe failed: Get "http://10.244.5.193:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Warning Unhealthy 40m event-controller Pod gaussdb-bpsmmq-gaussdb-1: Readiness probe failed: Get "http://10.244.5.73:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Warning Unhealthy 40m event-controller Pod gaussdb-bpsmmq-gaussdb-2: Readiness probe failed: command timed out: "bash -c LD_LIBRARY_PATH=$GAUSSHOME/lib:$LD_LIBRARY_PATH gsql -h 127.0.0.1 -p \"$***GAUSSDB_BASEPORT***\" -U $***GAUSSDB_USER*** -W \"$***GAUSSDB_PASSWORD***\" -dpostgres -c \"select 1\"\n" timed out after 5s Normal ComponentPhaseTransition 39m cluster-controller component is Failed Warning Failed 39m cluster-controller Cluster: gaussdb-bpsmmq is Failed, check according to the components message Warning Unhealthy 38m (x3 over 40m) event-controller Pod gaussdb-bpsmmq-gaussdb-0: Readiness probe failed: gsql: could not connect to server: Operation now in progress Is the server running on host "127.0.0.1" and accepting TCP/IP connections on port 5432? Warning Unhealthy 36m (x3 over 36m) event-controller Pod gaussdb-bpsmmq-gaussdb-0: Readiness probe failed: gsql: FATAL: can not accept connection in pending mode. Normal PreCheckSucceed 35m cluster-controller The operator has started the provisioning of Cluster: gaussdb-bpsmmq Normal ApplyResourcesSucceed 35m cluster-controller Successfully applied for resources Warning FailedScheduling 29m event-controller Pod gaussdb-bpsmmq-gaussdb-1: 0/3 nodes are available: 1 Insufficient memory, 1 node(s) exceed max volume count, 1 node(s) were unschedulable. preemption: 0/3 nodes are available: 1 Preemption is not helpful for scheduling, 2 No preemption victims found for incoming pod. Warning FailedScheduling 29m event-controller Pod gaussdb-bpsmmq-gaussdb-2: 0/3 nodes are available: 1 Insufficient memory, 1 node(s) exceed max volume count, 1 node(s) were unschedulable. preemption: 0/3 nodes are available: 1 Preemption is not helpful for scheduling, 2 No preemption victims found for incoming pod. Warning Abnormal 25m cluster-controller Cluster: gaussdb-bpsmmq is Abnormal, check according to the components message Normal ComponentPhaseTransition 25m cluster-controller component is Abnormal Warning Unhealthy 24m (x8 over 25m) event-controller Pod gaussdb-bpsmmq-gaussdb-0: Readiness probe failed: Get "http://10.244.1.59:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Normal Running 23m (x3 over 35m) cluster-controller Cluster: gaussdb-bpsmmq is ready, current phase is Running Normal ClusterReady 23m (x3 over 35m) cluster-controller Cluster: gaussdb-bpsmmq is ready, current phase is Running Normal AllReplicasReady 23m (x3 over 35m) cluster-controller all pods of components are ready, waiting for the probe detection successful Normal ComponentPhaseTransition 23m (x3 over 35m) cluster-controller component is Running Warning ComponentsNotReady 23m (x3 over 35m) cluster-controller pods are unavailable in Components: [gaussdb], refer to related component message in Cluster.status.components Warning ReplicasNotReady 8m28s (x6 over 35m) cluster-controller pods are not ready in Components: [gaussdb], refer to related component message in Cluster.status.components Normal ComponentPhaseTransition 8m28s (x7 over 39m) cluster-controller component is Updating ------------------------------------------------------------------------------------------------------------------ check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 600m / 600m 2254857830400m / 2254857830400m data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:19 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 600m / 600m 2254857830400m / 2254857830400m data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:24 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 600m / 600m 2254857830400m / 2254857830400m data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:18 UTC+0800 check pod status done check cluster status again check ops status `kbcli cluster list-ops gaussdb-bpsmmq --status all --namespace ns-ybzhw ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-bpsmmq-verticalscaling-rnbch ns-ybzhw VerticalScaling gaussdb-bpsmmq gaussdb Running 2/3 Sep 01,2025 12:17 UTC+0800 ops_status:gaussdb-bpsmmq-verticalscaling-rnbch ns-ybzhw VerticalScaling gaussdb-bpsmmq gaussdb Running 2/3 Sep 01,2025 12:17 UTC+0800 ops_status:gaussdb-bpsmmq-verticalscaling-rnbch ns-ybzhw VerticalScaling gaussdb-bpsmmq gaussdb Running 2/3 Sep 01,2025 12:17 UTC+0800 ops_status:gaussdb-bpsmmq-verticalscaling-rnbch ns-ybzhw VerticalScaling gaussdb-bpsmmq gaussdb Running 2/3 Sep 01,2025 12:17 UTC+0800 ops_status:gaussdb-bpsmmq-verticalscaling-rnbch ns-ybzhw VerticalScaling gaussdb-bpsmmq gaussdb Running 2/3 Sep 01,2025 12:17 UTC+0800 ops_status:gaussdb-bpsmmq-verticalscaling-rnbch ns-ybzhw VerticalScaling gaussdb-bpsmmq gaussdb Running 3/3 Sep 01,2025 12:17 UTC+0800 ops_status:gaussdb-bpsmmq-verticalscaling-rnbch ns-ybzhw VerticalScaling gaussdb-bpsmmq gaussdb Running 3/3 Sep 01,2025 12:17 UTC+0800 check ops status done ops_status:gaussdb-bpsmmq-verticalscaling-rnbch ns-ybzhw VerticalScaling gaussdb-bpsmmq gaussdb Succeed 3/3 Sep 01,2025 12:17 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests gaussdb-bpsmmq-verticalscaling-rnbch --namespace ns-ybzhw ` opsrequest.apps.kubeblocks.io/gaussdb-bpsmmq-verticalscaling-rnbch patched `kbcli cluster delete-ops --name gaussdb-bpsmmq-verticalscaling-rnbch --force --auto-approve --namespace ns-ybzhw ` OpsRequest gaussdb-bpsmmq-verticalscaling-rnbch deleted `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:23:13 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test failover podfailure check cluster status before cluster-failover-podfailure check cluster status done cluster_status:Running check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-gaussdb-bpsmmq" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-gaussdb-bpsmmq" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podfailure-gaussdb-bpsmmq namespace: ns-ybzhw spec: selector: namespaces: - ns-ybzhw labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-bpsmmq-gaussdb-0 mode: all action: pod-failure duration: 2m `kubectl apply -f test-chaos-mesh-podfailure-gaussdb-bpsmmq.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podfailure-gaussdb-bpsmmq created apply test-chaos-mesh-podfailure-gaussdb-bpsmmq.yaml Success `rm -rf test-chaos-mesh-podfailure-gaussdb-bpsmmq.yaml` podfailure chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Failed Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Failed cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "gaussdb-bpsmmq-gaussdb-1" force deleted cluster_status:Updating cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "gaussdb-bpsmmq-gaussdb-0" force deleted Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "gaussdb-bpsmmq-gaussdb-1" force deleted cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 600m / 600m 2254857830400m / 2254857830400m data:20Gi aks-cicdamdpool-38368475-vmss000008/10.224.0.9 Sep 01,2025 12:32 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 600m / 600m 2254857830400m / 2254857830400m data:20Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:32 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 600m / 600m 2254857830400m / 2254857830400m data:20Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:18 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-1;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-gaussdb-bpsmmq --namespace ns-ybzhw ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-gaussdb-bpsmmq" force deleted Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-gaussdb-bpsmmq" not found check failover pod name failover pod name:gaussdb-bpsmmq-gaussdb-1 failover podfailure Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:34:42 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success `kubectl get pvc -l app.kubernetes.io/instance=gaussdb-bpsmmq,apps.kubeblocks.io/component-name=gaussdb,apps.kubeblocks.io/vct-name=data --namespace ns-ybzhw ` cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in gaussdb-bpsmmq namespace. `kbcli cluster volume-expand gaussdb-bpsmmq --auto-approve --force=true --components gaussdb --volume-claim-templates data --storage 24Gi --namespace ns-ybzhw ` OpsRequest gaussdb-bpsmmq-volumeexpansion-8tjs4 created successfully, you can view the progress: kbcli cluster describe-ops gaussdb-bpsmmq-volumeexpansion-8tjs4 -n ns-ybzhw check ops status `kbcli cluster list-ops gaussdb-bpsmmq --status all --namespace ns-ybzhw ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-bpsmmq-volumeexpansion-8tjs4 ns-ybzhw VolumeExpansion gaussdb-bpsmmq gaussdb Running -/- Sep 01,2025 12:34 UTC+0800 check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 600m / 600m 2254857830400m / 2254857830400m data:24Gi aks-cicdamdpool-38368475-vmss000008/10.224.0.9 Sep 01,2025 12:32 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 600m / 600m 2254857830400m / 2254857830400m data:24Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:32 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 600m / 600m 2254857830400m / 2254857830400m data:24Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:18 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-1;secondary: gaussdb-bpsmmq-gaussdb-0 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done No resources found in gaussdb-bpsmmq namespace. check ops status `kbcli cluster list-ops gaussdb-bpsmmq --status all --namespace ns-ybzhw ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-bpsmmq-volumeexpansion-8tjs4 ns-ybzhw VolumeExpansion gaussdb-bpsmmq gaussdb Succeed 3/3 Sep 01,2025 12:34 UTC+0800 check ops status done ops_status:gaussdb-bpsmmq-volumeexpansion-8tjs4 ns-ybzhw VolumeExpansion gaussdb-bpsmmq gaussdb Succeed 3/3 Sep 01,2025 12:34 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests gaussdb-bpsmmq-volumeexpansion-8tjs4 --namespace ns-ybzhw ` opsrequest.apps.kubeblocks.io/gaussdb-bpsmmq-volumeexpansion-8tjs4 patched `kbcli cluster delete-ops --name gaussdb-bpsmmq-volumeexpansion-8tjs4 --force --auto-approve --namespace ns-ybzhw ` OpsRequest gaussdb-bpsmmq-volumeexpansion-8tjs4 deleted `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:42:24 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-1 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success test switchover cluster promote check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster promote gaussdb-bpsmmq --auto-approve --force=true --component gaussdb --namespace ns-ybzhw ` component:gaussdb OpsRequest gaussdb-bpsmmq-switchover-z4sdb created successfully, you can view the progress: kbcli cluster describe-ops gaussdb-bpsmmq-switchover-z4sdb -n ns-ybzhw check ops status `kbcli cluster list-ops gaussdb-bpsmmq --status all --namespace ns-ybzhw ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-bpsmmq-switchover-z4sdb ns-ybzhw Switchover gaussdb-bpsmmq gaussdb Running 0/1 Sep 01,2025 12:42 UTC+0800 check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Updating Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 600m / 600m 2254857830400m / 2254857830400m data:24Gi aks-cicdamdpool-38368475-vmss000008/10.224.0.9 Sep 01,2025 12:32 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 600m / 600m 2254857830400m / 2254857830400m data:24Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:32 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 600m / 600m 2254857830400m / 2254857830400m data:24Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:18 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-0;secondary: gaussdb-bpsmmq-gaussdb-1 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done check ops status `kbcli cluster list-ops gaussdb-bpsmmq --status all --namespace ns-ybzhw ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-bpsmmq-switchover-z4sdb ns-ybzhw Switchover gaussdb-bpsmmq gaussdb Running 0/1 Sep 01,2025 12:42 UTC+0800 check ops status done ops_status:gaussdb-bpsmmq-switchover-z4sdb ns-ybzhw Switchover gaussdb-bpsmmq gaussdb Succeed 1/1 Sep 01,2025 12:42 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests gaussdb-bpsmmq-switchover-z4sdb --namespace ns-ybzhw ` opsrequest.apps.kubeblocks.io/gaussdb-bpsmmq-switchover-z4sdb patched `kbcli cluster delete-ops --name gaussdb-bpsmmq-switchover-z4sdb --force --auto-approve --namespace ns-ybzhw ` OpsRequest gaussdb-bpsmmq-switchover-z4sdb deleted `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Mon Sep 1 04:42:59 UTC 2025 value ------- vzvtz (1 row) check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check db_client batch data count `echo "echo \"select count(*) from executions_loop_table;\" | gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d executions_loop -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check db_client batch data Success switchover pod:gaussdb-bpsmmq-gaussdb-0 switchover success cluster update terminationPolicy WipeOut `kbcli cluster update gaussdb-bpsmmq --termination-policy=WipeOut --namespace ns-ybzhw ` cluster.apps.kubeblocks.io/gaussdb-bpsmmq updated (no change) check cluster status `kbcli cluster list gaussdb-bpsmmq --show-labels --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-bpsmmq ns-ybzhw gaussdb WipeOut Running Sep 01,2025 11:24 UTC+0800 app.kubernetes.io/instance=gaussdb-bpsmmq,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-bpsmmq --namespace ns-ybzhw ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-bpsmmq-gaussdb-0 ns-ybzhw gaussdb-bpsmmq gaussdb Running primary 0 600m / 600m 2254857830400m / 2254857830400m data:24Gi aks-cicdamdpool-38368475-vmss000008/10.224.0.9 Sep 01,2025 12:32 UTC+0800 gaussdb-bpsmmq-gaussdb-1 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 600m / 600m 2254857830400m / 2254857830400m data:24Gi aks-cicdamdpool-38368475-vmss000006/10.224.0.10 Sep 01,2025 12:32 UTC+0800 gaussdb-bpsmmq-gaussdb-2 ns-ybzhw gaussdb-bpsmmq gaussdb Running secondary 0 600m / 600m 2254857830400m / 2254857830400m data:24Gi aks-cicdamdpool-38368475-vmss000000/10.224.0.5 Sep 01,2025 12:18 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-bpsmmq-gaussdb-0;secondary: gaussdb-bpsmmq-gaussdb-1 gaussdb-bpsmmq-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-bpsmmq` set secret: gaussdb-bpsmmq-gaussdb-account-dba `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-bpsmmq-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:511qH9i3MJ6g;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-bpsmmq-gaussdb.ns-ybzhw.svc.cluster.local -U dba -p 5432 -d postgres -W '511qH9i3MJ6g' " | kubectl exec -it gaussdb-bpsmmq-gaussdb-0 --namespace ns-ybzhw -- bash -c "su - omm"` check cluster connect done cluster list-logs `kbcli cluster list-logs gaussdb-bpsmmq --namespace ns-ybzhw ` No log files found. You can enable the log feature with the kbcli command below. kbcli cluster update gaussdb-bpsmmq --enable-all-logs=true --namespace ns-ybzhw Error from server (NotFound): pods "gaussdb-bpsmmq-gaussdb-0" not found cluster logs `kbcli cluster logs gaussdb-bpsmmq --tail 30 --namespace ns-ybzhw ` Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) expected instance path: [/gaussdb/volume_data/data/pg_hba.conf] gs_guc sethba: host all omm 10.244.5.136/32 trust: [/gaussdb/volume_data/data/pg_hba.conf] Total instances: 1. Failed instances: 0. Success to perform gs_guc! The gs_guc run with the following arguments: [gs_guc -D /gaussdb/volume_data/data -c pgxc_node_name='gaussdb_bpsmmq_gaussdb_0' set ]. expected instance path: [/gaussdb/volume_data/data/postgresql.conf] gs_guc set: pgxc_node_name='gaussdb_bpsmmq_gaussdb_0': [/gaussdb/volume_data/data/postgresql.conf] Total instances: 1. Failed instances: 0. Success to perform gs_guc! The gs_guc run with the following arguments: [gs_guc -D /gaussdb/volume_data/data -c port=5432 set ]. expected instance path: [/gaussdb/volume_data/data/postgresql.conf] gs_guc set: port=5432: [/gaussdb/volume_data/data/postgresql.conf] Total instances: 1. Failed instances: 0. Success to perform gs_guc! [2025-09-01 04:33:51.457][1360][][gs_ctl]: gs_ctl reload ,datadir is /gaussdb/volume_data/data [2025-09-01 04:33:51.461][1360][][gs_ctl]: could not send reload signal (PID: 1424): No such process % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed ***"output":"cluster info is up to date, no need to refresh"*** 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 198 100 60 100 138 28208 64880 --:--:-- --:--:-- --:--:-- 99000 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 138 0 0 100 138 0 114 0:00:01 0:00:01 --:--:-- 114 100 138 0 0 100 138 0 62 0:00:02 0:00:02 --:--:-- 62 100 138 0 0 100 138 0 43 0:00:03 0:00:03 --:--:-- 43 100 138 0 0 100 138 0 32 0:00:04 0:00:04 --:--:-- 32 100 198 100 60 100 138 12 28 0:00:05 0:00:04 0:00:01 40 100 198 100 60 100 138 12 28 0:00:05 0:00:04 0:00:01 16 ***"output":"cluster info is up to date, no need to refresh"*** delete cluster gaussdb-bpsmmq `kbcli cluster delete gaussdb-bpsmmq --auto-approve --namespace ns-ybzhw ` Cluster gaussdb-bpsmmq deleted pod_info:gaussdb-bpsmmq-gaussdb-0 4/4 Running 0 11m gaussdb-bpsmmq-gaussdb-1 4/4 Running 0 11m gaussdb-bpsmmq-gaussdb-2 4/4 Running 0 25m pod_info:gaussdb-bpsmmq-gaussdb-0 4/4 Terminating 0 11m gaussdb-bpsmmq-gaussdb-1 4/4 Terminating 0 11m gaussdb-bpsmmq-gaussdb-2 4/4 Terminating 0 26m No resources found in ns-ybzhw namespace. delete cluster pod done No resources found in ns-ybzhw namespace. check cluster resource non-exist OK: pvc No resources found in ns-ybzhw namespace. delete cluster done No resources found in ns-ybzhw namespace. No resources found in ns-ybzhw namespace. No resources found in ns-ybzhw namespace. GaussDB Test Suite All Done! --------------------------------------GaussDB (Topology = replication Replicas 3) Test Result-------------------------------------- [PASSED]|[Create]|[Topology=replication;ComponentVersion=gaussdb;ServiceVersion=2.23.1;]|[Description=Create a cluster with the specified topology replication and component version gaussdb and service version 2.23.1] [PASSED]|[Connect]|[ComponentName=gaussdb]|[Description=Connect to the cluster] [PASSED]|[AddData]|[Values=vzvtz]|[Description=Add data to the cluster] [PASSED]|[No-Failover]|[HA=Full CPU;Durations=2m;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience CPU full either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high CPU load.] [PASSED]|[No-Failover]|[HA=Network Delay;Durations=2m;ComponentName=gaussdb]|[Description=Simulates network delay fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to delay network.] [WARNING]|[CheckFailover]|[FailoverType=networkloss]|[Description=-] [PASSED]|[Failover]|[HA=Network Loss;Durations=2m;ComponentName=gaussdb]|[Description=Simulates network loss fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to loss network.] [PASSED]|[Failover]|[HA=Connection Stress;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience connection stress either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Connection load.] [WARNING]|[CheckFailover]|[FailoverType=networkcorrupt]|[Description=-] [PASSED]|[Failover]|[HA=Network Corrupt;Durations=2m;ComponentName=gaussdb]|[Description=Simulates network corrupt fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to corrupt network.] [PASSED]|[No-Failover]|[HA=Network Partition;Durations=2m;ComponentName=gaussdb]|[Description=Simulates network partition fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to partition network.] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [WARNING]|[CheckFailover]|[FailoverType=networkbandwidth]|[Description=-] [PASSED]|[Failover]|[HA=Network Bandwidth;Durations=2m;ComponentName=gaussdb]|[Description=Simulates network bandwidth fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to bandwidth network.] [PASSED]|[Update]|[Monitor=true]|[Description=Update the cluster Monitor enable] [PASSED]|[No-Failover]|[HA=DNS Random;Durations=2m;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience random IP addresses being returned by the DNS service for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to the DNS service returning random IP addresses.] [PASSED]|[Failover]|[HA=OOM;Durations=2m;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience OOM either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Memory load.] [PASSED]|[Failover]|[HA=Pod Kill;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience kill for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to kill.] [SKIPPED]|[Failover]|[HA=Evicting Pod;ComponentName=gaussdb]|[Description=Simulates conditions where pods evicting either due to node drained thereby testing the application's resilience to unavailability of some replicas due to evicting.] [PASSED]|[No-Failover]|[HA=Time Offset;Durations=2m;ComponentName=gaussdb]|[Description=Simulates a time offset scenario thereby testing the application's resilience to potential slowness/unavailability of some replicas due to time offset.] [PASSED]|[No-Failover]|[HA=Network Duplicate;Durations=2m;ComponentName=gaussdb]|[Description=Simulates network duplicate fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to duplicate network.] [PASSED]|[No-Failover]|[HA=DNS Error;Durations=2m;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience DNS service errors for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to DNS service errors.] [PASSED]|[No-Failover]|[HA=Kill 1;ComponentName=gaussdb]|[Description=Simulates conditions where process 1 killed either due to expected/undesired processes thereby testing the application's resilience to unavailability of some replicas due to abnormal termination signals.] [PASSED]|[Reconfiguring]|[ComponentName=gaussdb;alarm_report_interval=20]|[Description=Reconfiguring the cluster specify component gaussdb set alarm_report_interval=20] [PASSED]|[VerticalScaling]|[ComponentName=gaussdb]|[Description=VerticalScaling the cluster specify component gaussdb] [PASSED]|[Failover]|[HA=Pod Failure;Durations=2m;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience failure for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to failure.] [PASSED]|[VolumeExpansion]|[ComponentName=gaussdb]|[Description=VolumeExpansion the cluster specify component gaussdb] [PASSED]|[SwitchOver]|[ComponentName=gaussdb]|[Description=SwitchOver the cluster specify component gaussdb] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]