source commons files source engines files source kubeblocks files `kubectl get namespace | grep ns-qgdgh ` `kubectl create namespace ns-qgdgh` namespace/ns-qgdgh created create namespace ns-qgdgh done download kbcli `gh release list --repo apecloud/kbcli --limit 100 | (grep "0.9" || true)` `curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v0.9.4-beta.1` Your system is linux_amd64 Installing kbcli ... Downloading ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 32.1M 100 32.1M 0 0 94.6M 0 --:--:-- --:--:-- --:--:-- 94.6M kbcli installed successfully. Kubernetes: v1.32.5-eks-5d4a308 KubeBlocks: 0.9.4 kbcli: 0.9.4-beta.1 WARNING: version difference between kbcli (0.9.4-beta.1) and kubeblocks (0.9.4) Make sure your docker service is running and begin your journey with kbcli: kbcli playground init For more information on how to get started, please visit: https://kubeblocks.io download kbcli v0.9.4-beta.1 done Kubernetes: v1.32.5-eks-5d4a308 KubeBlocks: 0.9.4 kbcli: 0.9.4-beta.1 WARNING: version difference between kbcli (0.9.4-beta.1) and kubeblocks (0.9.4) Kubernetes Env: v1.32.5-eks-5d4a308 POD_RESOURCES: No resources found found default storage class: gp3 kubeblocks version is:0.9.4 skip upgrade kubeblocks Error: no repositories to show helm repo add chaos-mesh https://charts.chaos-mesh.org "chaos-mesh" has been added to your repositories add helm chart repo chaos-mesh success chaos mesh already installed check cluster definition set component name:gaussdb set component version set component version:gaussdb set service versions:2.23.1 set service versions sorted:2.23.1 no cluster version found set gaussdb component definition set gaussdb component definition gaussdb-repl set replicas first:3,2.23.1 set replicas third:3,2.23.1 set replicas fourth:3,2.23.1 set minimum cmpv service version set minimum cmpv service version replicas:3,2.23.1 REPORT_COUNT:1 CLUSTER_TOPOLOGY:replication topology replication found in cluster definition gaussdb LIMIT_CPU:0.5 LIMIT_MEMORY:2 storage size: 20 No resources found in ns-qgdgh namespace. termination_policy:DoNotTerminate create 3 replica DoNotTerminate gaussdb cluster check cluster definition check component definition set component definition by component version check cmpd by labels set component definition2: gaussdb-repl by component version:gaussdb apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: name: gaussdb-ofzmam namespace: ns-qgdgh annotations: kubeblocks.io/host-network: "gaussdb" spec: clusterDefinitionRef: gaussdb topology: replication terminationPolicy: DoNotTerminate componentSpecs: - name: gaussdb serviceVersion: 2.23.1 replicas: 3 resources: requests: cpu: 500m memory: 2Gi limits: cpu: 500m memory: 2Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi `kubectl apply -f test_create_gaussdb-ofzmam.yaml` cluster.apps.kubeblocks.io/gaussdb-ofzmam created apply test_create_gaussdb-ofzmam.yaml Success `rm -rf test_create_gaussdb-ofzmam.yaml` check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Jun 19,2025 18:16 UTC+0800 cluster_status: cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:16 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-6-5.us-west-2.compute.internal/172.31.6.5 Jun 19,2025 18:16 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-248.us-west-2.compute.internal/172.31.10.248 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-2;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-1 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-2 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check pod gaussdb-ofzmam-gaussdb-2 container_name gaussdb exist password A8Jj75xmB492 check pod gaussdb-ofzmam-gaussdb-2 container_name exporter exist password A8Jj75xmB492 check pod gaussdb-ofzmam-gaussdb-2 container_name lorry exist password A8Jj75xmB492 check pod gaussdb-ofzmam-gaussdb-2 container_name config-manager exist password A8Jj75xmB492 No container logs contain secret password. describe cluster `kbcli cluster describe gaussdb-ofzmam --namespace ns-qgdgh ` Name: gaussdb-ofzmam Created Time: Jun 19,2025 18:16 UTC+0800 NAMESPACE CLUSTER-DEFINITION VERSION STATUS TERMINATION-POLICY ns-qgdgh gaussdb Running DoNotTerminate Endpoints: COMPONENT MODE INTERNAL EXTERNAL gaussdb ReadWrite gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:8999 Topology: COMPONENT INSTANCE ROLE STATUS AZ NODE CREATED-TIME gaussdb gaussdb-ofzmam-gaussdb-0 secondary Running us-west-2a ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:16 UTC+0800 gaussdb gaussdb-ofzmam-gaussdb-1 secondary Running us-west-2a ip-172-31-6-5.us-west-2.compute.internal/172.31.6.5 Jun 19,2025 18:16 UTC+0800 gaussdb gaussdb-ofzmam-gaussdb-2 primary Running us-west-2a ip-172-31-10-248.us-west-2.compute.internal/172.31.10.248 Jun 19,2025 18:16 UTC+0800 Resources Allocation: COMPONENT DEDICATED CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS gaussdb false 500m / 500m 2Gi / 2Gi data:20Gi kb-default-sc Images: COMPONENT TYPE IMAGE gaussdb docker.io/apecloud/gaussdb:2.23.01.200 Data Protection: BACKUP-REPO AUTO-BACKUP BACKUP-SCHEDULE BACKUP-METHOD BACKUP-RETENTION RECOVERABLE-TIME Show cluster events: kbcli cluster list-events -n ns-qgdgh gaussdb-ofzmam `kbcli cluster label gaussdb-ofzmam app.kubernetes.io/instance- --namespace ns-qgdgh ` label "app.kubernetes.io/instance" not found. `kbcli cluster label gaussdb-ofzmam app.kubernetes.io/instance=gaussdb-ofzmam --namespace ns-qgdgh ` `kbcli cluster label gaussdb-ofzmam --list --namespace ns-qgdgh ` NAME NAMESPACE LABELS gaussdb-ofzmam ns-qgdgh app.kubernetes.io/instance=gaussdb-ofzmam clusterdefinition.kubeblocks.io/name=gaussdb clusterversion.kubeblocks.io/name= label cluster app.kubernetes.io/instance=gaussdb-ofzmam Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=gaussdb-ofzmam --namespace ns-qgdgh ` `kbcli cluster label gaussdb-ofzmam --list --namespace ns-qgdgh ` NAME NAMESPACE LABELS gaussdb-ofzmam ns-qgdgh app.kubernetes.io/instance=gaussdb-ofzmam case.name=kbcli.test1 clusterdefinition.kubeblocks.io/name=gaussdb clusterversion.kubeblocks.io/name= label cluster case.name=kbcli.test1 Success `kbcli cluster label gaussdb-ofzmam case.name=kbcli.test2 --overwrite --namespace ns-qgdgh ` `kbcli cluster label gaussdb-ofzmam --list --namespace ns-qgdgh ` NAME NAMESPACE LABELS gaussdb-ofzmam ns-qgdgh app.kubernetes.io/instance=gaussdb-ofzmam case.name=kbcli.test2 clusterdefinition.kubeblocks.io/name=gaussdb clusterversion.kubeblocks.io/name= label cluster case.name=kbcli.test2 Success `kbcli cluster label gaussdb-ofzmam case.name- --namespace ns-qgdgh ` `kbcli cluster label gaussdb-ofzmam --list --namespace ns-qgdgh ` NAME NAMESPACE LABELS gaussdb-ofzmam ns-qgdgh app.kubernetes.io/instance=gaussdb-ofzmam clusterdefinition.kubeblocks.io/name=gaussdb clusterversion.kubeblocks.io/name= delete cluster label case.name Success cluster connect `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"show max_connections;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-2 --namespace ns-qgdgh -- bash -c "su - omm" ` Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:20:06 UTC 2025 max_connections ----------------- 225 (1 row) connect cluster Success set max_connections to 225 insert batch data by db client Error from server (NotFound): pods "test-db-client-executionloop-gaussdb-ofzmam" not found DB_CLIENT_BATCH_DATA_COUNT: `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-gaussdb-ofzmam --namespace ns-qgdgh ` Error from server (NotFound): pods "test-db-client-executionloop-gaussdb-ofzmam" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-executionloop-gaussdb-ofzmam" not found `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres apiVersion: v1 kind: Pod metadata: name: test-db-client-executionloop-gaussdb-ofzmam namespace: ns-qgdgh spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local" - "--user" - "dba" - "--password" - "A8Jj75xmB492" - "--port" - "5432" - "--dbtype" - "gaussdb" - "--test" - "executionloop" - "--duration" - "60" - "--interval" - "1" restartPolicy: Never `kubectl apply -f test-db-client-executionloop-gaussdb-ofzmam.yaml` pod/test-db-client-executionloop-gaussdb-ofzmam created apply test-db-client-executionloop-gaussdb-ofzmam.yaml Success `rm -rf test-db-client-executionloop-gaussdb-ofzmam.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 6s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 11s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 17s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 23s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 28s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 34s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 40s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 46s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 52s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 58s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 64s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 70s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 76s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 82s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 88s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 94s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 100s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 106s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 112s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 118s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 2m3s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 2m9s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 2m15s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 2m21s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 2m27s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 2m33s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 2m39s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 2m45s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 2m51s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 2m57s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 3m3s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 3m9s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 3m15s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 3m21s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 3m27s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 3m33s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 3m38s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 3m44s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 3m50s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 3m56s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 4m2s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 4m8s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 4m14s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 4m20s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 4m26s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 4m32s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 4m38s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 4m44s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 4m50s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 4m56s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 5m2s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 5m8s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 5m14s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 5m20s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 5m26s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 5m32s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 5m38s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 5m44s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 5m50s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 5m56s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-gaussdb-ofzmam 1/1 Running 0 6m2s check pod test-db-client-executionloop-gaussdb-ofzmam status timeout --------------------------------------get pod test-db-client-executionloop-gaussdb-ofzmam yaml-------------------------------------- `kubectl get pod test-db-client-executionloop-gaussdb-ofzmam -o yaml --namespace ns-qgdgh ` apiVersion: v1 kind: Pod metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | ***"apiVersion":"v1","kind":"Pod","metadata":***"annotations":***,"name":"test-db-client-executionloop-gaussdb-ofzmam","namespace":"ns-qgdgh"***,"spec":***"containers":[***"args":["--host","gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local","--user","dba","--password","A8Jj75xmB492","--port","5432","--dbtype","gaussdb","--test","executionloop","--duration","60","--interval","1"],"image":"docker.io/apecloud/dbclient:test","imagePullPolicy":"IfNotPresent","name":"test-dbclient"***],"restartPolicy":"Never"*** creationTimestamp: "2025-06-19T10:20:51Z" name: test-db-client-executionloop-gaussdb-ofzmam namespace: ns-qgdgh resourceVersion: "24228" uid: d920e4cf-66ad-4d97-88df-ef4f850b0a6d spec: containers: - args: - --host - gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local - --user - dba - --password - A8Jj75xmB492 - --port - "5432" - --dbtype - gaussdb - --test - executionloop - --duration - "60" - --interval - "1" image: docker.io/apecloud/dbclient:test imagePullPolicy: IfNotPresent name: test-dbclient resources: *** terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: kube-api-access-4v6lm readOnly: true dnsPolicy: ClusterFirst enableServiceLinks: true nodeName: ip-172-31-2-181.us-west-2.compute.internal preemptionPolicy: PreemptLowerPriority priority: 0 restartPolicy: Never schedulerName: default-scheduler securityContext: *** serviceAccount: default serviceAccountName: default terminationGracePeriodSeconds: 30 tolerations: - effect: NoExecute key: node.kubernetes.io/not-ready operator: Exists tolerationSeconds: 300 - effect: NoExecute key: node.kubernetes.io/unreachable operator: Exists tolerationSeconds: 300 volumes: - name: kube-api-access-4v6lm projected: defaultMode: 420 sources: - serviceAccountToken: expirationSeconds: 3607 path: token - configMap: items: - key: ca.crt path: ca.crt name: kube-root-ca.crt - downwardAPI: items: - fieldRef: apiVersion: v1 fieldPath: metadata.namespace path: namespace status: conditions: - lastProbeTime: null lastTransitionTime: "2025-06-19T10:20:53Z" status: "True" type: PodReadyToStartContainers - lastProbeTime: null lastTransitionTime: "2025-06-19T10:20:51Z" status: "True" type: Initialized - lastProbeTime: null lastTransitionTime: "2025-06-19T10:20:53Z" status: "True" type: Ready - lastProbeTime: null lastTransitionTime: "2025-06-19T10:20:53Z" status: "True" type: ContainersReady - lastProbeTime: null lastTransitionTime: "2025-06-19T10:20:51Z" status: "True" type: PodScheduled containerStatuses: - containerID: containerd://d93a3b950327404c22c1b157448d407ba7d8b69e9567b4f312e1efa1114ecd57 image: docker.io/apecloud/dbclient:test imageID: docker.io/apecloud/dbclient@sha256:94dcfdb6d1e9c728bba32050d67228b14611badb037c61c1740ebfc68677d43f lastState: *** name: test-dbclient ready: true restartCount: 0 started: true state: running: startedAt: "2025-06-19T10:20:52Z" volumeMounts: - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: kube-api-access-4v6lm readOnly: true recursiveReadOnly: Disabled hostIP: 172.31.2.181 hostIPs: - ip: 172.31.2.181 phase: Running podIP: 172.31.0.205 podIPs: - ip: 172.31.0.205 qosClass: BestEffort startTime: "2025-06-19T10:20:51Z" ------------------------------------------------------------------------------------------------------------------ --------------------------------------describe pod test-db-client-executionloop-gaussdb-ofzmam-------------------------------------- `kubectl describe pod test-db-client-executionloop-gaussdb-ofzmam --namespace ns-qgdgh ` Name: test-db-client-executionloop-gaussdb-ofzmam Namespace: ns-qgdgh Priority: 0 Service Account: default Node: ip-172-31-2-181.us-west-2.compute.internal/172.31.2.181 Start Time: Thu, 19 Jun 2025 18:20:51 +0800 Labels: Annotations: Status: Running IP: 172.31.0.205 IPs: IP: 172.31.0.205 Containers: test-dbclient: Container ID: containerd://d93a3b950327404c22c1b157448d407ba7d8b69e9567b4f312e1efa1114ecd57 Image: docker.io/apecloud/dbclient:test Image ID: docker.io/apecloud/dbclient@sha256:94dcfdb6d1e9c728bba32050d67228b14611badb037c61c1740ebfc68677d43f Port: Host Port: Args: --host gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local --user dba --password A8Jj75xmB492 --port 5432 --dbtype gaussdb --test executionloop --duration 60 --interval 1 State: Running Started: Thu, 19 Jun 2025 18:20:52 +0800 Ready: True Restart Count: 0 Environment: Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-4v6lm (ro) Conditions: Type Status PodReadyToStartContainers True Initialized True Ready True ContainersReady True PodScheduled True Volumes: kube-api-access-4v6lm: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: BestEffort Node-Selectors: Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 6m5s default-scheduler Successfully assigned ns-qgdgh/test-db-client-executionloop-gaussdb-ofzmam to ip-172-31-2-181.us-west-2.compute.internal Normal Pulled 6m4s kubelet Container image "docker.io/apecloud/dbclient:test" already present on machine Normal Created 6m4s kubelet Created container: test-dbclient Normal Started 6m4s kubelet Started container test-dbclient ------------------------------------------------------------------------------------------------------------------ --------------------------------------pod test-db-client-executionloop-gaussdb-ofzmam-------------------------------------- `kubectl logs test-db-client-executionloop-gaussdb-ofzmam --namespace ns-qgdgh --tail 500` --host gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local --user dba --password A8Jj75xmB492 --port 5432 --dbtype gaussdb --test executionloop --duration 60 --interval 1 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider] 10:20:53.150 [main] DEBUG com.clickhouse.jdbc.ClickHouseDriver -- ClickHouse Driver 0.0.0.0(JDBC: 0.0.0.0) registered Jun 19, 2025 10:20:53 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [5b802920-0188-49d3-bf6c-9e05e17f7363] Try to connect. IP: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 Jun 19, 2025 10:20:54 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [172.31.0.205:49688/aussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local/10.100.100.69:5432] Connection is established. ID: 5b802920-0188-49d3-bf6c-9e05e17f7363 Jun 19, 2025 10:20:54 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: Connect complete. ID: 5b802920-0188-49d3-bf6c-9e05e17f7363 Execution loop start: create databases executions_loop CREATE DATABASE executions_loop; reconnect connection executions_loop Jun 19, 2025 10:20:55 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [bf7867c6-4da1-4a44-b425-059205d9638d] Try to connect. IP: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 Jun 19, 2025 10:20:55 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [172.31.0.205:35798/aussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local/10.100.100.69:5432] Connection is established. ID: bf7867c6-4da1-4a44-b425-059205d9638d Jun 19, 2025 10:20:55 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: Connect complete. ID: bf7867c6-4da1-4a44-b425-059205d9638d drop table executions_loop_table DROP TABLE IF EXISTS executions_loop_table; create table executions_loop_table CREATE TABLE IF NOT EXISTS executions_loop_table (id SERIAL PRIMARY KEY , value text); Execution loop start:INSERT INTO executions_loop_table (value) VALUES ('executions_loop_test_1'); [ 1s ] executions total: 1 successful: 1 failed: 0 disconnect: 0 [ 2s ] executions total: 61 successful: 61 failed: 0 disconnect: 0 [ 3s ] executions total: 69 successful: 69 failed: 0 disconnect: 0 ------------------------------------------------------------------------------------------------------------------ check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:16 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-6-5.us-west-2.compute.internal/172.31.6.5 Jun 19,2025 18:16 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-248.us-west-2.compute.internal/172.31.10.248 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-2;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-1 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-2 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done --host gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local --user dba --password A8Jj75xmB492 --port 5432 --dbtype gaussdb --test executionloop --duration 60 --interval 1 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider] 10:20:53.150 [main] DEBUG com.clickhouse.jdbc.ClickHouseDriver -- ClickHouse Driver 0.0.0.0(JDBC: 0.0.0.0) registered Jun 19, 2025 10:20:53 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [5b802920-0188-49d3-bf6c-9e05e17f7363] Try to connect. IP: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 Jun 19, 2025 10:20:54 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [172.31.0.205:49688/aussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local/10.100.100.69:5432] Connection is established. ID: 5b802920-0188-49d3-bf6c-9e05e17f7363 Jun 19, 2025 10:20:54 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: Connect complete. ID: 5b802920-0188-49d3-bf6c-9e05e17f7363 Execution loop start: create databases executions_loop CREATE DATABASE executions_loop; reconnect connection executions_loop Jun 19, 2025 10:20:55 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [bf7867c6-4da1-4a44-b425-059205d9638d] Try to connect. IP: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 Jun 19, 2025 10:20:55 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [172.31.0.205:35798/aussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local/10.100.100.69:5432] Connection is established. ID: bf7867c6-4da1-4a44-b425-059205d9638d Jun 19, 2025 10:20:55 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: Connect complete. ID: bf7867c6-4da1-4a44-b425-059205d9638d drop table executions_loop_table DROP TABLE IF EXISTS executions_loop_table; create table executions_loop_table CREATE TABLE IF NOT EXISTS executions_loop_table (id SERIAL PRIMARY KEY , value text); Execution loop start:INSERT INTO executions_loop_table (value) VALUES ('executions_loop_test_1'); [ 1s ] executions total: 1 successful: 1 failed: 0 disconnect: 0 [ 2s ] executions total: 61 successful: 61 failed: 0 disconnect: 0 [ 3s ] executions total: 69 successful: 69 failed: 0 disconnect: 0 DB_CLIENT_BATCH_DATA_COUNT: `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-gaussdb-ofzmam --namespace ns-qgdgh ` pod/test-db-client-executionloop-gaussdb-ofzmam patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-executionloop-gaussdb-ofzmam" force deleted `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"DROP TABLE IF EXISTS tmp_table; CREATE TABLE IF NOT EXISTS tmp_table (id INT PRIMARY KEY , value text); INSERT INTO tmp_table (id,value) VALUES (1,'kmlut');\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-2 --namespace ns-qgdgh -- bash -c "su - omm"` Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:27:11 UTC 2025 NOTICE: table "tmp_table" does not exist, skipping NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "tmp_table_pkey" for table "tmp_table" DROP TABLE CREATE TABLE INSERT 0 1 add consistent data kmlut Success test failover networkloss check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkloss-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkloss-gaussdb-ofzmam" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkloss-gaussdb-ofzmam" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkloss-gaussdb-ofzmam namespace: ns-qgdgh spec: selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-2 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkloss-gaussdb-ofzmam.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkloss-gaussdb-ofzmam created apply test-chaos-mesh-networkloss-gaussdb-ofzmam.yaml Success `rm -rf test-chaos-mesh-networkloss-gaussdb-ofzmam.yaml` networkloss chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:16 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-6-5.us-west-2.compute.internal/172.31.6.5 Jun 19,2025 18:16 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-248.us-west-2.compute.internal/172.31.10.248 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-2;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-1 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-2 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkloss-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkloss-gaussdb-ofzmam" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networkloss-gaussdb-ofzmam patched check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-2 failover networkloss Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-2 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:29:54 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover networkcorrupt check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorrupt-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorrupt-gaussdb-ofzmam" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorrupt-gaussdb-ofzmam" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorrupt-gaussdb-ofzmam namespace: ns-qgdgh spec: selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-2 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorrupt-gaussdb-ofzmam.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorrupt-gaussdb-ofzmam created apply test-chaos-mesh-networkcorrupt-gaussdb-ofzmam.yaml Success `rm -rf test-chaos-mesh-networkcorrupt-gaussdb-ofzmam.yaml` networkcorrupt chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:16 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-6-5.us-west-2.compute.internal/172.31.6.5 Jun 19,2025 18:16 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-248.us-west-2.compute.internal/172.31.10.248 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-2;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-1 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-2 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorrupt-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorrupt-gaussdb-ofzmam" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorrupt-gaussdb-ofzmam patched check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-2 failover networkcorrupt Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-2 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:32:27 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success cluster vscale check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale gaussdb-ofzmam --auto-approve --force=true --components gaussdb --cpu 600m --memory 2.1Gi --namespace ns-qgdgh ` OpsRequest gaussdb-ofzmam-verticalscaling-8rcp5 created successfully, you can view the progress: kbcli cluster describe-ops gaussdb-ofzmam-verticalscaling-8rcp5 -n ns-qgdgh check ops status `kbcli cluster list-ops gaussdb-ofzmam --status all --namespace ns-qgdgh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-ofzmam-verticalscaling-8rcp5 ns-qgdgh VerticalScaling gaussdb-ofzmam gaussdb Running 0/3 Jun 19,2025 18:32 UTC+0800 check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:20Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:35 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:20Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:33 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:20Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:36 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-0;secondary: gaussdb-ofzmam-gaussdb-1 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done check ops status `kbcli cluster list-ops gaussdb-ofzmam --status all --namespace ns-qgdgh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-ofzmam-verticalscaling-8rcp5 ns-qgdgh VerticalScaling gaussdb-ofzmam gaussdb Succeed 3/3 Jun 19,2025 18:32 UTC+0800 check ops status done ops_status:gaussdb-ofzmam-verticalscaling-8rcp5 ns-qgdgh VerticalScaling gaussdb-ofzmam gaussdb Succeed 3/3 Jun 19,2025 18:32 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests gaussdb-ofzmam-verticalscaling-8rcp5 --namespace ns-qgdgh ` opsrequest.apps.kubeblocks.io/gaussdb-ofzmam-verticalscaling-8rcp5 patched `kbcli cluster delete-ops --name gaussdb-ofzmam-verticalscaling-8rcp5 --force --auto-approve --namespace ns-qgdgh ` OpsRequest gaussdb-ofzmam-verticalscaling-8rcp5 deleted `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:38:33 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover dnserror check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-gaussdb-ofzmam" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-gaussdb-ofzmam" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnserror-gaussdb-ofzmam namespace: ns-qgdgh spec: selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-0 mode: all action: error duration: 2m `kubectl apply -f test-chaos-mesh-dnserror-gaussdb-ofzmam.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnserror-gaussdb-ofzmam created apply test-chaos-mesh-dnserror-gaussdb-ofzmam.yaml Success `rm -rf test-chaos-mesh-dnserror-gaussdb-ofzmam.yaml` dnserror chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:20Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:35 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:20Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:33 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:20Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:36 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-0;secondary: gaussdb-ofzmam-gaussdb-1 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-gaussdb-ofzmam" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-gaussdb-ofzmam" not found check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-0 failover dnserror Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:41:06 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success `kubectl get pvc -l app.kubernetes.io/instance=gaussdb-ofzmam,apps.kubeblocks.io/component-name=gaussdb,apps.kubeblocks.io/vct-name=data --namespace ns-qgdgh ` cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in gaussdb-ofzmam namespace. `kbcli cluster volume-expand gaussdb-ofzmam --auto-approve --force=true --components gaussdb --volume-claim-templates data --storage 22Gi --namespace ns-qgdgh ` OpsRequest gaussdb-ofzmam-volumeexpansion-dxh82 created successfully, you can view the progress: kbcli cluster describe-ops gaussdb-ofzmam-volumeexpansion-dxh82 -n ns-qgdgh check ops status `kbcli cluster list-ops gaussdb-ofzmam --status all --namespace ns-qgdgh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-ofzmam-volumeexpansion-dxh82 ns-qgdgh VolumeExpansion gaussdb-ofzmam gaussdb Running 0/3 Jun 19,2025 18:41 UTC+0800 check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:35 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:33 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:36 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-0;secondary: gaussdb-ofzmam-gaussdb-1 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done No resources found in gaussdb-ofzmam namespace. check ops status `kbcli cluster list-ops gaussdb-ofzmam --status all --namespace ns-qgdgh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-ofzmam-volumeexpansion-dxh82 ns-qgdgh VolumeExpansion gaussdb-ofzmam gaussdb Succeed 3/3 Jun 19,2025 18:41 UTC+0800 check ops status done ops_status:gaussdb-ofzmam-volumeexpansion-dxh82 ns-qgdgh VolumeExpansion gaussdb-ofzmam gaussdb Succeed 3/3 Jun 19,2025 18:41 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests gaussdb-ofzmam-volumeexpansion-dxh82 --namespace ns-qgdgh ` opsrequest.apps.kubeblocks.io/gaussdb-ofzmam-volumeexpansion-dxh82 patched `kbcli cluster delete-ops --name gaussdb-ofzmam-volumeexpansion-dxh82 --force --auto-approve --namespace ns-qgdgh ` OpsRequest gaussdb-ofzmam-volumeexpansion-dxh82 deleted `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:42:23 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover delete pod:gaussdb-ofzmam-gaussdb-0 `kubectl delete pod gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh ` pod "gaussdb-ofzmam-gaussdb-0" deleted check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:43 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:33 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:36 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-1 failover Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:44:25 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover connectionstressover check node drain check node drain success Error from server (NotFound): pods "test-db-client-connectionstressover-gaussdb-ofzmam" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstressover-gaussdb-ofzmam --namespace ns-qgdgh ` Error from server (NotFound): pods "test-db-client-connectionstressover-gaussdb-ofzmam" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-connectionstressover-gaussdb-ofzmam" not found `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres apiVersion: v1 kind: Pod metadata: name: test-db-client-connectionstressover-gaussdb-ofzmam namespace: ns-qgdgh spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local" - "--user" - "dba" - "--password" - "A8Jj75xmB492" - "--port" - "5432" - "--database" - "postgres" - "--dbtype" - "gaussdb" - "--test" - "connectionstress" - "--connections" - "225" - "--duration" - "60" restartPolicy: Never `kubectl apply -f test-db-client-connectionstressover-gaussdb-ofzmam.yaml` pod/test-db-client-connectionstressover-gaussdb-ofzmam created apply test-db-client-connectionstressover-gaussdb-ofzmam.yaml Success `rm -rf test-db-client-connectionstressover-gaussdb-ofzmam.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstressover-gaussdb-ofzmam 1/1 Running 0 6s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstressover-gaussdb-ofzmam 1/1 Running 0 11s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstressover-gaussdb-ofzmam 1/1 Running 0 17s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstressover-gaussdb-ofzmam 1/1 Running 0 23s check pod test-db-client-connectionstressover-gaussdb-ofzmam status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstressover-gaussdb-ofzmam 0/1 Completed 0 29s check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:43 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:33 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:36 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-0;secondary: gaussdb-ofzmam-gaussdb-1 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at com.huawei.opengauss.jdbc.core.PGStream.(PGStream.java:70) at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:119) at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:246) ... 11 more Jun 19, 2025 10:45:10 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [29e9cbdf-5367-45ee-b6c8-dae6afa58159] Try to connect. IP: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 10:45:10.026 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:opengauss://gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432/postgres?useSSL=false Jun 19, 2025 10:45:10 AM org.opengauss.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [7eeb7d5e-f6e7-4ff8-8d99-0c05ca0b3c61] Try to connect. IP: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 Failed to connect to GaussDB database: com.huawei.opengauss.jdbc.util.PSQLException: Connection to gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. Trying with database GaussDB. Jun 19, 2025 10:45:10 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [8485a880-e807-486e-84db-cc3d8c81f6c9] Try to connect. IP: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 10:45:10.029 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:opengauss://gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432/postgres?useSSL=false Jun 19, 2025 10:45:10 AM org.opengauss.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [abfef6e4-b3dd-4510-b7a1-08231a3361c2] Try to connect. IP: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 java.io.IOException: Failed to connect to GaussDB database: at com.apecloud.dbtester.tester.GaussdbTester.connect(GaussdbTester.java:62) at com.apecloud.dbtester.tester.GaussdbTester.connectionStress(GaussdbTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.huawei.opengauss.jdbc.util.PSQLException: Connection to gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:370) at com.huawei.opengauss.jdbc.core.ConnectionFactory.openConnection(ConnectionFactory.java:57) at com.huawei.opengauss.jdbc.jdbc.PgConnection.(PgConnection.java:274) at com.huawei.opengauss.jdbc.Driver.makeConnection(Driver.java:576) at com.huawei.opengauss.jdbc.Driver.connect(Driver.java:315) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.GaussdbTester.connect(GaussdbTester.java:60) ... 4 more Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at com.huawei.opengauss.jdbc.core.PGStream.(PGStream.java:70) at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:119) at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:246) ... 11 more Jun 19, 2025 10:45:10 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [7d8cc9d7-88fa-4a49-95ba-18670e04d32d] Try to connect. IP: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 10:45:10.036 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:opengauss://gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432/postgres?useSSL=false Jun 19, 2025 10:45:10 AM org.opengauss.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [ea89d2d6-c2e6-4f1f-a5e6-8969b0b3a180] Try to connect. IP: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 Failed to connect to GaussDB database: com.huawei.opengauss.jdbc.util.PSQLException: Connection to gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. Trying with database GaussDB. Jun 19, 2025 10:45:10 AM com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [4792f71e-a12e-461a-aa4d-00192dff6b54] Try to connect. IP: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 10:45:10.039 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:opengauss://gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432/postgres?useSSL=false Jun 19, 2025 10:45:10 AM org.opengauss.core.v3.ConnectionFactoryImpl openConnectionImpl INFO: [1daaf1dd-2e89-4b72-9439-deb4713bcd41] Try to connect. IP: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 java.io.IOException: Failed to connect to GaussDB database: at com.apecloud.dbtester.tester.GaussdbTester.connect(GaussdbTester.java:62) at com.apecloud.dbtester.tester.GaussdbTester.connectionStress(GaussdbTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.huawei.opengauss.jdbc.util.PSQLException: Connection to gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:370) at com.huawei.opengauss.jdbc.core.ConnectionFactory.openConnection(ConnectionFactory.java:57) at com.huawei.opengauss.jdbc.jdbc.PgConnection.(PgConnection.java:274) at com.huawei.opengauss.jdbc.Driver.makeConnection(Driver.java:576) at com.huawei.opengauss.jdbc.Driver.connect(Driver.java:315) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.GaussdbTester.connect(GaussdbTester.java:60) ... 4 more Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at com.huawei.opengauss.jdbc.core.PGStream.(PGStream.java:70) at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:119) at com.huawei.opengauss.jdbc.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:246) ... 11 more Test Result: null Connection Information: Database Type: gaussdb Host: gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local Port: 5432 Database: postgres Table: User: dba Org: Access Mode: mysql Test Type: connectionstress Connection Count: 225 Duration: 60 seconds `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstressover-gaussdb-ofzmam --namespace ns-qgdgh ` pod/test-db-client-connectionstressover-gaussdb-ofzmam patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-connectionstressover-gaussdb-ofzmam" force deleted check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-0 failover connectionstressover Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:45:35 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover kill1 check node drain check node drain success `kill 1` Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file exec return message: check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:43 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:33 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:36 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-0;secondary: gaussdb-ofzmam-gaussdb-1 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-0 failover kill1 Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:46:17 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test switchover cluster promote check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster promote gaussdb-ofzmam --auto-approve --force=true --component gaussdb --namespace ns-qgdgh ` component:gaussdb OpsRequest gaussdb-ofzmam-switchover-4bbrm created successfully, you can view the progress: kbcli cluster describe-ops gaussdb-ofzmam-switchover-4bbrm -n ns-qgdgh check ops status `kbcli cluster list-ops gaussdb-ofzmam --status all --namespace ns-qgdgh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-ofzmam-switchover-4bbrm ns-qgdgh Switchover gaussdb-ofzmam gaussdb Running 0/1 Jun 19,2025 18:46 UTC+0800 check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:43 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:33 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:36 UTC+0800 check pod status done check cluster role No resources found in ns-qgdgh namespace. primary: ;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-1 gaussdb-ofzmam-gaussdb-2 No resources found in ns-qgdgh namespace. primary: ;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-1 gaussdb-ofzmam-gaussdb-2 check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done check ops status `kbcli cluster list-ops gaussdb-ofzmam --status all --namespace ns-qgdgh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-ofzmam-switchover-4bbrm ns-qgdgh Switchover gaussdb-ofzmam gaussdb Running 0/1 Jun 19,2025 18:46 UTC+0800 check ops status done ops_status:gaussdb-ofzmam-switchover-4bbrm ns-qgdgh Switchover gaussdb-ofzmam gaussdb Succeed 1/1 Jun 19,2025 18:46 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests gaussdb-ofzmam-switchover-4bbrm --namespace ns-qgdgh ` opsrequest.apps.kubeblocks.io/gaussdb-ofzmam-switchover-4bbrm patched `kbcli cluster delete-ops --name gaussdb-ofzmam-switchover-4bbrm --force --auto-approve --namespace ns-qgdgh ` OpsRequest gaussdb-ofzmam-switchover-4bbrm deleted `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:47:04 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success switchover pod:gaussdb-ofzmam-gaussdb-1 switchover success test failover networkpartition check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-gaussdb-ofzmam" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-gaussdb-ofzmam" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkpartition-gaussdb-ofzmam namespace: ns-qgdgh spec: selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-1 action: partition mode: all target: mode: all selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-0 direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkpartition-gaussdb-ofzmam.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkpartition-gaussdb-ofzmam created apply test-chaos-mesh-networkpartition-gaussdb-ofzmam.yaml Success `rm -rf test-chaos-mesh-networkpartition-gaussdb-ofzmam.yaml` networkpartition chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:43 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:33 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:36 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-gaussdb-ofzmam" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networkpartition-gaussdb-ofzmam patched check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-1 failover networkpartition Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:50:11 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success cluster configure component_tmp: gaussdb check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster configure gaussdb-ofzmam --auto-approve --force=true --set alarm_report_interval=20 --components gaussdb --config-spec gaussdb-config --config-file postgresql.conf --namespace ns-qgdgh ` Will updated configure file meta: ConfigSpec: gaussdb-config ConfigFile: postgresql.conf ComponentName: gaussdb ClusterName: gaussdb-ofzmam OpsRequest gaussdb-ofzmam-reconfiguring-kjr52 created successfully, you can view the progress: kbcli cluster describe-ops gaussdb-ofzmam-reconfiguring-kjr52 -n ns-qgdgh check ops status `kbcli cluster list-ops gaussdb-ofzmam --status all --namespace ns-qgdgh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-ofzmam-reconfiguring-kjr52 ns-qgdgh Reconfiguring gaussdb-ofzmam gaussdb Running -/- Jun 19,2025 18:50 UTC+0800 check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:43 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:33 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:36 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done check ops status `kbcli cluster list-ops gaussdb-ofzmam --status all --namespace ns-qgdgh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-ofzmam-reconfiguring-kjr52 ns-qgdgh Reconfiguring gaussdb-ofzmam gaussdb Succeed 3/3 Jun 19,2025 18:50 UTC+0800 check ops status done ops_status:gaussdb-ofzmam-reconfiguring-kjr52 ns-qgdgh Reconfiguring gaussdb-ofzmam gaussdb Succeed 3/3 Jun 19,2025 18:50 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests gaussdb-ofzmam-reconfiguring-kjr52 --namespace ns-qgdgh ` opsrequest.apps.kubeblocks.io/gaussdb-ofzmam-reconfiguring-kjr52 patched `kbcli cluster delete-ops --name gaussdb-ofzmam-reconfiguring-kjr52 --force --auto-approve --namespace ns-qgdgh ` OpsRequest gaussdb-ofzmam-reconfiguring-kjr52 deleted check config variables Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file current value_actual: 20 configure:[alarm_report_interval] result actual:[20] equal expected:[20] `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:51:03 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover networkdelay check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-gaussdb-ofzmam" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-gaussdb-ofzmam" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkdelay-gaussdb-ofzmam namespace: ns-qgdgh spec: selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-1 mode: all action: delay delay: latency: 2000ms correlation: '100' jitter: 0ms direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkdelay-gaussdb-ofzmam.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkdelay-gaussdb-ofzmam created apply test-chaos-mesh-networkdelay-gaussdb-ofzmam.yaml Success `rm -rf test-chaos-mesh-networkdelay-gaussdb-ofzmam.yaml` networkdelay chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:43 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:33 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:36 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-gaussdb-ofzmam" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networkdelay-gaussdb-ofzmam patched check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-1 failover networkdelay Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 10:53:31 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover timeoffset check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-gaussdb-ofzmam" not found Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-gaussdb-ofzmam" not found apiVersion: chaos-mesh.org/v1alpha1 kind: TimeChaos metadata: name: test-chaos-mesh-timeoffset-gaussdb-ofzmam namespace: ns-qgdgh spec: selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-1 mode: all timeOffset: '-10m' clockIds: - CLOCK_REALTIME duration: 2m `kubectl apply -f test-chaos-mesh-timeoffset-gaussdb-ofzmam.yaml` timechaos.chaos-mesh.org/test-chaos-mesh-timeoffset-gaussdb-ofzmam created apply test-chaos-mesh-timeoffset-gaussdb-ofzmam.yaml Success `rm -rf test-chaos-mesh-timeoffset-gaussdb-ofzmam.yaml` timeoffset chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:43 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-6-5.us-west-2.compute.internal/172.31.6.5 Jun 19,2025 18:53 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:36 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-0;secondary: gaussdb-ofzmam-gaussdb-1 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. timechaos.chaos-mesh.org/test-chaos-mesh-timeoffset-gaussdb-ofzmam patched check failover pod name timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-gaussdb-ofzmam" force deleted failover pod name:gaussdb-ofzmam-gaussdb-0 failover timeoffset Failure cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart gaussdb-ofzmam --auto-approve --force=true --namespace ns-qgdgh ` OpsRequest gaussdb-ofzmam-restart-xmxhq created successfully, you can view the progress: kbcli cluster describe-ops gaussdb-ofzmam-restart-xmxhq -n ns-qgdgh check ops status `kbcli cluster list-ops gaussdb-ofzmam --status all --namespace ns-qgdgh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-ofzmam-restart-xmxhq ns-qgdgh Restart gaussdb-ofzmam gaussdb Running 0/3 Jun 19,2025 18:56 UTC+0800 check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:59 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:58 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:56 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done check ops status `kbcli cluster list-ops gaussdb-ofzmam --status all --namespace ns-qgdgh ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME gaussdb-ofzmam-restart-xmxhq ns-qgdgh Restart gaussdb-ofzmam gaussdb Succeed 3/3 Jun 19,2025 18:56 UTC+0800 check ops status done ops_status:gaussdb-ofzmam-restart-xmxhq ns-qgdgh Restart gaussdb-ofzmam gaussdb Succeed 3/3 Jun 19,2025 18:56 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests gaussdb-ofzmam-restart-xmxhq --namespace ns-qgdgh ` opsrequest.apps.kubeblocks.io/gaussdb-ofzmam-restart-xmxhq patched `kbcli cluster delete-ops --name gaussdb-ofzmam-restart-xmxhq --force --auto-approve --namespace ns-qgdgh ` OpsRequest gaussdb-ofzmam-restart-xmxhq deleted `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 11:01:00 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover networkbandwidth check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidth-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidth-gaussdb-ofzmam" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidth-gaussdb-ofzmam" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkbandwidth-gaussdb-ofzmam namespace: ns-qgdgh spec: selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-1 action: bandwidth mode: all bandwidth: rate: '1bps' limit: 20971520 buffer: 10000 duration: 2m `kubectl apply -f test-chaos-mesh-networkbandwidth-gaussdb-ofzmam.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkbandwidth-gaussdb-ofzmam created apply test-chaos-mesh-networkbandwidth-gaussdb-ofzmam.yaml Success `rm -rf test-chaos-mesh-networkbandwidth-gaussdb-ofzmam.yaml` networkbandwidth chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:59 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:58 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:56 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidth-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidth-gaussdb-ofzmam" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networkbandwidth-gaussdb-ofzmam patched check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-1 failover networkbandwidth Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 11:03:33 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover podfailure check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-gaussdb-ofzmam" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-gaussdb-ofzmam" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podfailure-gaussdb-ofzmam namespace: ns-qgdgh spec: selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-1 mode: all action: pod-failure duration: 2m `kubectl apply -f test-chaos-mesh-podfailure-gaussdb-ofzmam.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podfailure-gaussdb-ofzmam created apply test-chaos-mesh-podfailure-gaussdb-ofzmam.yaml Success `rm -rf test-chaos-mesh-podfailure-gaussdb-ofzmam.yaml` podfailure chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:59 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:58 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:56 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-2;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-1 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-2 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-gaussdb-ofzmam" force deleted Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-gaussdb-ofzmam" not found check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-2 failover podfailure Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-2 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 11:06:51 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover oom check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-gaussdb-ofzmam" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-gaussdb-ofzmam" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-gaussdb-ofzmam namespace: ns-qgdgh spec: selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-2 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-gaussdb-ofzmam.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-gaussdb-ofzmam created apply test-chaos-mesh-oom-gaussdb-ofzmam.yaml Success `rm -rf test-chaos-mesh-oom-gaussdb-ofzmam.yaml` check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 18:59 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:58 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:56 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-0;secondary: gaussdb-ofzmam-gaussdb-1 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-gaussdb-ofzmam" force deleted stresschaos.chaos-mesh.org/test-chaos-mesh-oom-gaussdb-ofzmam patched check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-0 failover oom Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 11:08:22 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover drainnode check node drain check node drain success kubectl get pod gaussdb-ofzmam-gaussdb-0 --namespace ns-qgdgh -o jsonpath='***.spec.nodeName***' get node name:ip-172-31-13-32.us-west-2.compute.internal success check if multiple pods are on the same node kubectl get pod gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -o jsonpath='***.spec.nodeName***' get node name:ip-172-31-10-206.us-west-2.compute.internal success kubectl get pod gaussdb-ofzmam-gaussdb-2 --namespace ns-qgdgh -o jsonpath='***.spec.nodeName***' get node name:ip-172-31-1-34.us-west-2.compute.internal success kubectl drain ip-172-31-13-32.us-west-2.compute.internal --delete-emptydir-data --ignore-daemonsets --force --grace-period 0 --timeout 60s node/ip-172-31-13-32.us-west-2.compute.internal cordoned Warning: ignoring DaemonSet-managed Pods: chaos-mesh/chaos-daemon-db2fj, kb-mqtcr/kb-addon-apecloud-otel-collector-jvh6n, kube-system/aws-node-k4wbp, kube-system/ebs-csi-node-85fqt, kube-system/kube-proxy-frgj5 evicting pod ns-qgdgh/gaussdb-ofzmam-gaussdb-0 evicting pod kb-mqtcr/kubeblocks-dataprotection-74c8cb94c5-2pfsd pod/kubeblocks-dataprotection-74c8cb94c5-2pfsd evicted pod/gaussdb-ofzmam-gaussdb-0 evicted node/ip-172-31-13-32.us-west-2.compute.internal drained kubectl uncordon ip-172-31-13-32.us-west-2.compute.internal node/ip-172-31-13-32.us-west-2.compute.internal uncordoned check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-97.us-west-2.compute.internal/172.31.1.97 Jun 19,2025 19:08 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:58 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:56 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-1 failover drainnode Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 11:10:26 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success cluster update monitor KUBEBLOCKS NAMESPACE:kb-mqtcr get kubeblocks namespace done check addon prometheus enabled check addon prometheus enabled done check addon grafana enabled check addon grafana enabled done check addon apecloud-otel-collector enabled check addon apecloud-otel-collector enabled done kubectl patch cluster gaussdb-ofzmam --namespace ns-qgdgh --type json -p '[***"op": "replace", "path": "/spec/componentSpecs/0/disableExporter", "value": false***]' cluster.apps.kubeblocks.io/gaussdb-ofzmam patched check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-97.us-west-2.compute.internal/172.31.1.97 Jun 19,2025 19:08 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:58 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:56 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done create curl-jq pod gaussdb-ofzmam-kb-curl `kubectl run --namespace kb-mqtcr gaussdb-ofzmam-kb-curl --restart='Never' --image docker.io/apecloud/curl-jq:0.1.0 --command -- sleep infinity ` pod/gaussdb-ofzmam-kb-curl created check pod status pod_status:NAME READY STATUS RESTARTS AGE gaussdb-ofzmam-kb-curl 0/1 ContainerCreating 0 1s check pod gaussdb-ofzmam-kb-curl status done `kubectl wait --for=condition=Ready pod gaussdb-ofzmam-kb-curl --namespace kb-mqtcr` pod/gaussdb-ofzmam-kb-curl condition met pod_status:NAME READY STATUS RESTARTS AGE gaussdb-ofzmam-kb-curl 1/1 Running 0 6s kubectl exec -it gaussdb-ofzmam-kb-curl --namespace kb-mqtcr -- curl -G "http://kb-addon-prometheus-server:80/api/v1/query" -d "query=sum(k8s_pod_cpu_limit)" `kubectl exec -it gaussdb-ofzmam-kb-curl --namespace kb-mqtcr -- curl -G "http://kb-addon-prometheus-server:80/api/v1/query" -d "query=sum(k8s_pod_cpu_limit)"` Unable to use a TTY - input is not a terminal or the right kind of file % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 122 100 122 0 0 15709 0 --:--:-- --:--:-- --:--:-- 17428 monitor_result:***"status":"success","data":***"resultType":"vector","result":[***"metric":***,"value":[1750331472.142,"25.300000000000004"]***]*** k8s_pod_cpu_limit: 1750331472.142 monitor cluster Success `kubectl delete pod gaussdb-ofzmam-kb-curl --namespace kb-mqtcr --force` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "gaussdb-ofzmam-kb-curl" force deleted check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-97.us-west-2.compute.internal/172.31.1.97 Jun 19,2025 19:08 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:58 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:56 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done test failover dnsrandom check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-gaussdb-ofzmam" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-gaussdb-ofzmam" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnsrandom-gaussdb-ofzmam namespace: ns-qgdgh spec: selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-1 mode: all action: random duration: 2m `kubectl apply -f test-chaos-mesh-dnsrandom-gaussdb-ofzmam.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnsrandom-gaussdb-ofzmam created apply test-chaos-mesh-dnsrandom-gaussdb-ofzmam.yaml Success `rm -rf test-chaos-mesh-dnsrandom-gaussdb-ofzmam.yaml` dnsrandom chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-97.us-west-2.compute.internal/172.31.1.97 Jun 19,2025 19:08 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:58 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:56 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-gaussdb-ofzmam" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-gaussdb-ofzmam" not found check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-1 failover dnsrandom Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 11:13:52 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover fullcpu check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-gaussdb-ofzmam" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-gaussdb-ofzmam" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-fullcpu-gaussdb-ofzmam namespace: ns-qgdgh spec: selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-1 mode: all stressors: cpu: workers: 100 load: 100 duration: 2m `kubectl apply -f test-chaos-mesh-fullcpu-gaussdb-ofzmam.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-fullcpu-gaussdb-ofzmam created apply test-chaos-mesh-fullcpu-gaussdb-ofzmam.yaml Success `rm -rf test-chaos-mesh-fullcpu-gaussdb-ofzmam.yaml` fullcpu chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-97.us-west-2.compute.internal/172.31.1.97 Jun 19,2025 19:08 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:58 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:56 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-gaussdb-ofzmam" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-gaussdb-ofzmam" not found check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-1 failover fullcpu Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 11:16:40 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success test failover networkduplicate check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-gaussdb-ofzmam" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-gaussdb-ofzmam" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkduplicate-gaussdb-ofzmam namespace: ns-qgdgh spec: selector: namespaces: - ns-qgdgh labelSelectors: apps.kubeblocks.io/pod-name: gaussdb-ofzmam-gaussdb-1 mode: all action: duplicate duplicate: duplicate: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkduplicate-gaussdb-ofzmam.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkduplicate-gaussdb-ofzmam created apply test-chaos-mesh-networkduplicate-gaussdb-ofzmam.yaml Success `rm -rf test-chaos-mesh-networkduplicate-gaussdb-ofzmam.yaml` networkduplicate chaos test waiting 120 seconds check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb DoNotTerminate Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-97.us-west-2.compute.internal/172.31.1.97 Jun 19,2025 19:08 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:58 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:56 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-gaussdb-ofzmam --namespace ns-qgdgh ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-gaussdb-ofzmam" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networkduplicate-gaussdb-ofzmam patched check failover pod name failover pod name:gaussdb-ofzmam-gaussdb-1 failover networkduplicate Success `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres `echo "echo \"SELECT value FROM tmp_table WHERE id = 1;\" | gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check data: Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) Unable to use a TTY - input is not a terminal or the right kind of file Last login: Thu Jun 19 11:19:11 UTC 2025 value ------- kmlut (1 row) check cluster data consistent Success cluster update terminationPolicy WipeOut `kbcli cluster update gaussdb-ofzmam --termination-policy=WipeOut --namespace ns-qgdgh ` cluster.apps.kubeblocks.io/gaussdb-ofzmam updated check cluster status `kbcli cluster list gaussdb-ofzmam --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam ns-qgdgh gaussdb WipeOut Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=gaussdb-ofzmam,clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-gaussdb-0 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-97.us-west-2.compute.internal/172.31.1.97 Jun 19,2025 19:08 UTC+0800 gaussdb-ofzmam-gaussdb-1 ns-qgdgh gaussdb-ofzmam gaussdb Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:58 UTC+0800 gaussdb-ofzmam-gaussdb-2 ns-qgdgh gaussdb-ofzmam gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-1-34.us-west-2.compute.internal/172.31.1.34 Jun 19,2025 18:56 UTC+0800 check pod status done check cluster role check cluster role done primary: gaussdb-ofzmam-gaussdb-1;secondary: gaussdb-ofzmam-gaussdb-0 gaussdb-ofzmam-gaussdb-2 `kubectl get secrets -l app.kubernetes.io/instance=gaussdb-ofzmam` set secret: gaussdb-ofzmam-gaussdb-account-dba `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.username***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.password***"` `kubectl get secrets gaussdb-ofzmam-gaussdb-account-dba -o jsonpath="***.data.port***"` DB_USERNAME:dba;DB_PASSWORD:A8Jj75xmB492;DB_PORT:5432;DB_DATABASE:postgres check cluster connect `echo "gsql -h gaussdb-ofzmam-gaussdb.ns-qgdgh.svc.cluster.local -U dba -p 5432 -d postgres -W 'A8Jj75xmB492' " | kubectl exec -it gaussdb-ofzmam-gaussdb-1 --namespace ns-qgdgh -- bash -c "su - omm"` check cluster connect done cluster gaussdb-roach backup `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.name***"` `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.namespace***"` `kubectl get secrets kb-backuprepo-w7bzj -n kb-mqtcr -o jsonpath="***.data.accessKeyId***"` `kubectl get secrets kb-backuprepo-w7bzj -n kb-mqtcr -o jsonpath="***.data.secretAccessKey***"` KUBEBLOCKS NAMESPACE:kb-mqtcr get kubeblocks namespace done `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-mqtcr -o jsonpath="***.items[0].data.root-user***"` `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-mqtcr -o jsonpath="***.items[0].data.root-password***"` minio_user:kbclitest,minio_password:kbclitest,minio_endpoint:kbcli-test-minio.kb-mqtcr.svc.cluster.local:9000 list minio bucket kbcli-test `echo 'mc config host add minioserver http://kbcli-test-minio.kb-mqtcr.svc.cluster.local:9000 kbclitest kbclitest;mc ls minioserver' | kubectl exec -it kbcli-test-minio-5c88d6b598-9v8l5 --namespace kb-mqtcr -- bash` Unable to use a TTY - input is not a terminal or the right kind of file list minio bucket done default backuprepo:backuprepo-kbcli-test exists `kbcli cluster backup gaussdb-ofzmam --method gaussdb-roach --namespace ns-qgdgh ` Backup backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 created successfully, you can view the progress: kbcli cluster list-backups --name=backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 -n ns-qgdgh check backup status `kbcli cluster list-backups gaussdb-ofzmam --namespace ns-qgdgh ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION CREATE-TIME COMPLETION-TIME EXPIRATION backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 ns-qgdgh gaussdb-ofzmam gaussdb-roach Running Jun 19,2025 19:19 UTC+0800 backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running backup_status:gaussdb-ofzmam-gaussdb-roach-Running check backup status done backup_status:backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 ns-qgdgh gaussdb-ofzmam gaussdb-roach Completed 21638721 114s Jun 19,2025 19:19 UTC+0800 Jun 19,2025 19:21 UTC+0800 cluster restore backup Error from server (NotFound): opsrequests.apps.kubeblocks.io "gaussdb-ofzmam-backup" not found `kbcli cluster describe-backup backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 --namespace ns-qgdgh ` Name: backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 Cluster: gaussdb-ofzmam Namespace: ns-qgdgh Spec: Method: gaussdb-roach Policy Name: gaussdb-ofzmam-gaussdb-backup-policy Status: Phase: Completed Total Size: 21638721 ActionSet Name: gaussdb-roach Repository: backuprepo-kbcli-test Duration: 114s Start Time: Jun 19,2025 19:19 UTC+0800 Completion Time: Jun 19,2025 19:21 UTC+0800 Path: /ns-qgdgh/gaussdb-ofzmam-7261c25a-4744-40af-8387-7ccdd64ce3fa/gaussdb/backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 Warning Events: `kbcli cluster restore gaussdb-ofzmam-backup --backup backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 --namespace ns-qgdgh ` Cluster gaussdb-ofzmam-backup created check cluster status `kbcli cluster list gaussdb-ofzmam-backup --show-labels --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS gaussdb-ofzmam-backup ns-qgdgh gaussdb WipeOut Jun 19,2025 19:21 UTC+0800 clusterdefinition.kubeblocks.io/name=gaussdb,clusterversion.kubeblocks.io/name= cluster_status: cluster_status: cluster_status: cluster_status: cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Failed cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating [Error] check cluster status timeout --------------------------------------get cluster gaussdb-ofzmam-backup yaml-------------------------------------- `kubectl get cluster gaussdb-ofzmam-backup -o yaml --namespace ns-qgdgh ` apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: annotations: kubeblocks.io/host-network: gaussdb kubeblocks.io/reconcile: "2025-06-19T11:30:17.601678923Z" kubeblocks.io/restore-from-backup: '***"gaussdb":***"doReadyRestoreAfterClusterRunning":"false","name":"backup-ns-qgdgh-gaussdb-ofzmam-20250619191956","namespace":"ns-qgdgh","volumeRestorePolicy":"Parallel"***' creationTimestamp: "2025-06-19T11:21:59Z" finalizers: - cluster.kubeblocks.io/finalizer generation: 1 labels: clusterdefinition.kubeblocks.io/name: gaussdb clusterversion.kubeblocks.io/name: "" name: gaussdb-ofzmam-backup namespace: ns-qgdgh resourceVersion: "97220" uid: 1ea71e3d-a51e-4a81-b64f-8d623946bb62 spec: clusterDefinitionRef: gaussdb componentSpecs: - componentDef: gaussdb-repl disableExporter: false name: gaussdb replicas: 3 resources: limits: cpu: 600m memory: 2254857830400m requests: cpu: 600m memory: 2254857830400m serviceVersion: 2.23.1 volumeClaimTemplates: - name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 22Gi resources: cpu: "0" memory: "0" storage: size: "0" terminationPolicy: WipeOut topology: replication status: clusterDefGeneration: 1 components: gaussdb: message: InstanceSet/gaussdb-ofzmam-backup-gaussdb: '["gaussdb-ofzmam-backup-gaussdb-2"]' phase: Updating podsReady: false conditions: - lastTransitionTime: "2025-06-19T11:21:59Z" message: 'The operator has started the provisioning of Cluster: gaussdb-ofzmam-backup' observedGeneration: 1 reason: PreCheckSucceed status: "True" type: ProvisioningStarted - lastTransitionTime: "2025-06-19T11:21:59Z" message: Successfully applied for resources observedGeneration: 1 reason: ApplyResourcesSucceed status: "True" type: ApplyResources - lastTransitionTime: "2025-06-19T11:22:00Z" message: 'pods are not ready in Components: [gaussdb], refer to related component message in Cluster.status.components' reason: ReplicasNotReady status: "False" type: ReplicasReady - lastTransitionTime: "2025-06-19T11:22:00Z" message: 'pods are unavailable in Components: [gaussdb], refer to related component message in Cluster.status.components' reason: ComponentsNotReady status: "False" type: Ready observedGeneration: 1 phase: Updating ------------------------------------------------------------------------------------------------------------------ --------------------------------------describe cluster gaussdb-ofzmam-backup-------------------------------------- `kubectl describe cluster gaussdb-ofzmam-backup --namespace ns-qgdgh ` Name: gaussdb-ofzmam-backup Namespace: ns-qgdgh Labels: clusterdefinition.kubeblocks.io/name=gaussdb clusterversion.kubeblocks.io/name= Annotations: kubeblocks.io/host-network: gaussdb kubeblocks.io/reconcile: 2025-06-19T11:30:17.601678923Z kubeblocks.io/restore-from-backup: ***"gaussdb":***"doReadyRestoreAfterClusterRunning":"false","name":"backup-ns-qgdgh-gaussdb-ofzmam-20250619191956","namespace":"ns-qgdgh","vol... API Version: apps.kubeblocks.io/v1alpha1 Kind: Cluster Metadata: Creation Timestamp: 2025-06-19T11:21:59Z Finalizers: cluster.kubeblocks.io/finalizer Generation: 1 Resource Version: 97220 UID: 1ea71e3d-a51e-4a81-b64f-8d623946bb62 Spec: Cluster Definition Ref: gaussdb Component Specs: Component Def: gaussdb-repl Disable Exporter: false Name: gaussdb Replicas: 3 Resources: Limits: Cpu: 600m Memory: 2254857830400m Requests: Cpu: 600m Memory: 2254857830400m Service Version: 2.23.1 Volume Claim Templates: Name: data Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 22Gi Resources: Cpu: 0 Memory: 0 Storage: Size: 0 Termination Policy: WipeOut Topology: replication Status: Cluster Def Generation: 1 Components: Gaussdb: Message: InstanceSet/gaussdb-ofzmam-backup-gaussdb: ["gaussdb-ofzmam-backup-gaussdb-2"] Phase: Updating Pods Ready: false Conditions: Last Transition Time: 2025-06-19T11:21:59Z Message: The operator has started the provisioning of Cluster: gaussdb-ofzmam-backup Observed Generation: 1 Reason: PreCheckSucceed Status: True Type: ProvisioningStarted Last Transition Time: 2025-06-19T11:21:59Z Message: Successfully applied for resources Observed Generation: 1 Reason: ApplyResourcesSucceed Status: True Type: ApplyResources Last Transition Time: 2025-06-19T11:22:00Z Message: pods are not ready in Components: [gaussdb], refer to related component message in Cluster.status.components Reason: ReplicasNotReady Status: False Type: ReplicasReady Last Transition Time: 2025-06-19T11:22:00Z Message: pods are unavailable in Components: [gaussdb], refer to related component message in Cluster.status.components Reason: ComponentsNotReady Status: False Type: Ready Observed Generation: 1 Phase: Updating Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal PreCheckSucceed 8m23s cluster-controller The operator has started the provisioning of Cluster: gaussdb-ofzmam-backup Normal ApplyResourcesSucceed 8m23s cluster-controller Successfully applied for resources Normal NeedWaiting 7m56s (x7 over 8m23s) component-controller waiting for restore "gaussdb-ofzmam-backup-gaussdb-1ea71e3d-preparedata" successfully Normal ComponentPhaseTransition 7m54s cluster-controller component is Creating Normal ComponentPhaseTransition 7m24s cluster-controller component is Failed Warning Failed 7m24s cluster-controller Cluster: gaussdb-ofzmam-backup is Failed, check according to the components message Normal ComponentPhaseTransition 7m22s cluster-controller component is Updating Warning Unhealthy 6m35s event-controller Pod gaussdb-ofzmam-backup-gaussdb-1: Readiness probe failed: gsql: could not connect to server: Operation now in progress Is the server running on host "127.0.0.1" and accepting TCP/IP connections on port 1038? Warning Unhealthy 6m34s event-controller Pod gaussdb-ofzmam-backup-gaussdb-0: Readiness probe failed: gsql: could not connect to server: Operation now in progress Is the server running on host "127.0.0.1" and accepting TCP/IP connections on port 1038? Warning Unhealthy 2m14s (x9 over 5m22s) event-controller Pod gaussdb-ofzmam-backup-gaussdb-0: Readiness probe failed: gsql: FATAL: can not accept connection in pending mode. Warning Unhealthy 2m6s (x7 over 4m22s) event-controller Pod gaussdb-ofzmam-backup-gaussdb-2: Readiness probe failed: gsql: FATAL: can not accept connection in pending mode. Warning Unhealthy 89s (x9 over 5m7s) event-controller Pod gaussdb-ofzmam-backup-gaussdb-1: Readiness probe failed: gsql: FATAL: can not accept connection in pending mode. ------------------------------------------------------------------------------------------------------------------ check pod status `kbcli cluster list-instances gaussdb-ofzmam-backup --namespace ns-qgdgh ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME gaussdb-ofzmam-backup-gaussdb-0 ns-qgdgh gaussdb-ofzmam-backup gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-5-144.us-west-2.compute.internal/172.31.5.144 Jun 19,2025 19:22 UTC+0800 gaussdb-ofzmam-backup-gaussdb-1 ns-qgdgh gaussdb-ofzmam-backup gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-3-13.us-west-2.compute.internal/172.31.3.13 Jun 19,2025 19:22 UTC+0800 gaussdb-ofzmam-backup-gaussdb-2 ns-qgdgh gaussdb-ofzmam-backup gaussdb Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:22Gi ip-172-31-13-32.us-west-2.compute.internal/172.31.13.32 Jun 19,2025 19:22 UTC+0800 check pod status done `kbcli cluster describe-backup backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 --namespace ns-qgdgh ` Name: backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 Cluster: gaussdb-ofzmam Namespace: ns-qgdgh Spec: Method: gaussdb-roach Policy Name: gaussdb-ofzmam-gaussdb-backup-policy Status: Phase: Completed Total Size: 21638721 ActionSet Name: gaussdb-roach Repository: backuprepo-kbcli-test Duration: 114s Start Time: Jun 19,2025 19:19 UTC+0800 Completion Time: Jun 19,2025 19:21 UTC+0800 Path: /ns-qgdgh/gaussdb-ofzmam-7261c25a-4744-40af-8387-7ccdd64ce3fa/gaussdb/backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 Warning Events: delete cluster gaussdb-ofzmam-backup `kbcli cluster delete gaussdb-ofzmam-backup --auto-approve --namespace ns-qgdgh ` Cluster gaussdb-ofzmam-backup deleted pod_info:gaussdb-ofzmam-backup-gaussdb-0 3/4 Running 0 8m3s gaussdb-ofzmam-backup-gaussdb-1 3/4 Running 0 8m3s gaussdb-ofzmam-backup-gaussdb-2 3/4 Running 1 (7m32s ago) 8m3s pod_info:gaussdb-ofzmam-backup-gaussdb-0 3/4 Terminating 0 8m24s gaussdb-ofzmam-backup-gaussdb-1 3/4 Terminating 0 8m24s gaussdb-ofzmam-backup-gaussdb-2 2/4 Terminating 1 (7m53s ago) 8m24s No resources found in ns-qgdgh namespace. delete cluster pod done No resources found in ns-qgdgh namespace. check cluster resource non-exist OK: pvc No resources found in ns-qgdgh namespace. delete cluster done No resources found in ns-qgdgh namespace. No resources found in ns-qgdgh namespace. No resources found in ns-qgdgh namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 --namespace ns-qgdgh ` backup.dataprotection.kubeblocks.io/backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 patched `kbcli cluster delete-backup gaussdb-ofzmam --name backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 --force --auto-approve --namespace ns-qgdgh ` Backup backup-ns-qgdgh-gaussdb-ofzmam-20250619191956 deleted No opsrequests found in ns-qgdgh namespace. cluster list-logs `kbcli cluster list-logs gaussdb-ofzmam --namespace ns-qgdgh ` No log files found. You can enable the log feature with the kbcli command below. kbcli cluster update gaussdb-ofzmam --enable-all-logs=true --namespace ns-qgdgh Error from server (NotFound): pods "gaussdb-ofzmam-gaussdb-1" not found cluster logs `kbcli cluster logs gaussdb-ofzmam --tail 30 --namespace ns-qgdgh ` Defaulted container "gaussdb" out of: gaussdb, exporter, lorry, config-manager, init-lorry (init) expected instance path: [/gaussdb/volume_data/data/pg_hba.conf] gs_guc sethba: host all omm 172.31.1.34/32 trust: [/gaussdb/volume_data/data/pg_hba.conf] Total instances: 1. Failed instances: 0. Success to perform gs_guc! The gs_guc run with the following arguments: [gs_guc -D /gaussdb/volume_data/data -c pgxc_node_name='gaussdb_ofzmam_gaussdb_1' set ]. expected instance path: [/gaussdb/volume_data/data/postgresql.conf] gs_guc set: pgxc_node_name='gaussdb_ofzmam_gaussdb_1': [/gaussdb/volume_data/data/postgresql.conf] Total instances: 1. Failed instances: 0. Success to perform gs_guc! The gs_guc run with the following arguments: [gs_guc -D /gaussdb/volume_data/data -c port=1025 set ]. expected instance path: [/gaussdb/volume_data/data/postgresql.conf] gs_guc set: port=1025: [/gaussdb/volume_data/data/postgresql.conf] Total instances: 1. Failed instances: 0. Success to perform gs_guc! [2025-06-19 11:05:58.134][1345][][gs_ctl]: gs_ctl reload ,datadir is /gaussdb/volume_data/data [2025-06-19 11:05:58.134][1345][][gs_ctl]: could not send reload signal (PID: 1406): No such process % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed ***"output":"cluster info is up to date, no need to refresh"*** 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 199 100 60 100 139 38910 90142 --:--:-- --:--:-- --:--:-- 194k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed ***"output":"cluster info is up to date, no need to refresh"*** 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 199 100 60 100 139 47468 107k --:--:-- --:--:-- --:--:-- 194k delete cluster gaussdb-ofzmam `kbcli cluster delete gaussdb-ofzmam --auto-approve --namespace ns-qgdgh ` Cluster gaussdb-ofzmam deleted pod_info:gaussdb-ofzmam-gaussdb-0 4/4 Running 0 22m gaussdb-ofzmam-gaussdb-1 4/4 Running 16 (25m ago) 33m gaussdb-ofzmam-gaussdb-2 4/4 Running 1 (24m ago) 34m pod_info:gaussdb-ofzmam-gaussdb-0 4/4 Terminating 0 23m gaussdb-ofzmam-gaussdb-1 4/4 Terminating 16 (26m ago) 33m gaussdb-ofzmam-gaussdb-2 4/4 Terminating 1 (24m ago) 35m No resources found in ns-qgdgh namespace. delete cluster pod done No resources found in ns-qgdgh namespace. check cluster resource non-exist OK: pvc No resources found in ns-qgdgh namespace. delete cluster done No resources found in ns-qgdgh namespace. No resources found in ns-qgdgh namespace. No resources found in ns-qgdgh namespace. GaussDB Test Suite All Done! --------------------------------------GaussDB (Topology = replication Replicas 3) Test Result-------------------------------------- [PASSED]|[Create]|[Topology=replication;ComponentVersion=gaussdb;ServiceVersion=2.23.1;]|[Description=Create a cluster with the specified topology replication and component version gaussdb and service version 2.23.1] [PASSED]|[Connect]|[ComponentName=gaussdb]|[Description=Connect to the cluster] [PASSED]|[AddData]|[Values=kmlut]|[Description=Add data to the cluster] [PASSED]|[Failover]|[HA=Network Loss;Durations=2m;ComponentName=gaussdb]|[Description=Simulates network loss fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to loss network.] [PASSED]|[Failover]|[HA=Network Corrupt;Durations=2m;ComponentName=gaussdb]|[Description=Simulates network corrupt fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to corrupt network.] [PASSED]|[VerticalScaling]|[ComponentName=gaussdb]|[Description=VerticalScaling the cluster specify component gaussdb] [PASSED]|[Failover]|[HA=DNS Error;Durations=2m;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience DNS service errors for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to DNS service errors.] [PASSED]|[VolumeExpansion]|[ComponentName=gaussdb]|[Description=VolumeExpansion the cluster specify component gaussdb] [PASSED]|[Failover]|[HA=Delete Pod;ComponentName=gaussdb]|[Description=Simulates conditions where pods terminating forced/graceful thereby testing deployment sanity (replica availability & uninterrupted service) and recovery workflow of the application.] [PASSED]|[Failover]|[HA=Connection Stress Failover;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience connection stress either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Connection load.] [PASSED]|[Failover]|[HA=Kill 1;ComponentName=gaussdb]|[Description=Simulates conditions where process 1 killed either due to expected/undesired processes thereby testing the application's resilience to unavailability of some replicas due to abnormal termination signals.] [PASSED]|[SwitchOver]|[ComponentName=gaussdb]|[Description=SwitchOver the cluster specify component gaussdb] [PASSED]|[Failover]|[HA=Network Partition;Durations=2m;ComponentName=gaussdb]|[Description=Simulates network partition fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to partition network.] [PASSED]|[Reconfiguring]|[ComponentName=gaussdb;alarm_report_interval=20]|[Description=Reconfiguring the cluster specify component gaussdb set alarm_report_interval=20] [PASSED]|[Failover]|[HA=Network Delay;Durations=2m;ComponentName=gaussdb]|[Description=Simulates network delay fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to delay network.] [FAILED]|[Failover]|[HA=Time Offset;Durations=2m;ComponentName=gaussdb]|[Description=Simulates a time offset scenario thereby testing the application's resilience to potential slowness/unavailability of some replicas due to time offset.] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [PASSED]|[Failover]|[HA=Network Bandwidth;Durations=2m;ComponentName=gaussdb]|[Description=Simulates network bandwidth fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to bandwidth network.] [PASSED]|[Failover]|[HA=Pod Failure;Durations=2m;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience failure for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to failure.] [PASSED]|[Failover]|[HA=OOM;Durations=2m;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience OOM either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Memory load.] [PASSED]|[Failover]|[HA=Evicting Pod;ComponentName=gaussdb]|[Description=Simulates conditions where pods evicting either due to node drained thereby testing the application's resilience to unavailability of some replicas due to evicting.] [PASSED]|[Check Monitor]|[Enable=true]|[Description=Check Monitor Enable] [PASSED]|[Update]|[Monitor=true]|[Description=Update the cluster Monitor enable] [PASSED]|[Failover]|[HA=DNS Random;Durations=2m;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience random IP addresses being returned by the DNS service for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to the DNS service returning random IP addresses.] [PASSED]|[Failover]|[HA=Full CPU;Durations=2m;ComponentName=gaussdb]|[Description=Simulates conditions where pods experience CPU full either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high CPU load.] [PASSED]|[Failover]|[HA=Network Duplicate;Durations=2m;ComponentName=gaussdb]|[Description=Simulates network duplicate fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to duplicate network.] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Backup]|[BackupMethod=gaussdb-roach]|[Description=The cluster gaussdb-roach Backup] [FAILED]|[Restore]|[BackupMethod=gaussdb-roach]|[Description=The cluster gaussdb-roach Restore] [PASSED]|[Delete Restore Cluster]|[BackupMethod=gaussdb-roach]|[Description=Delete the gaussdb-roach restore cluster] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]