bash test/kbcli/test_kbcli_0.9.sh --type 18 --version 0.9.5 --generate-output true --chaos-mesh true --drain-node true --random-namespace true --region eastus --cloud-provider aks CURRENT_TEST_DIR:test/kbcli source commons files source engines files source kubeblocks files `kubectl get namespace | grep ns-eicpq ` `kubectl create namespace ns-eicpq` namespace/ns-eicpq created create namespace ns-eicpq done download kbcli `gh release list --repo apecloud/kbcli --limit 100 | (grep "0.9" || true)` `curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v0.9.5-beta.8` Your system is linux_amd64 Installing kbcli ... Downloading ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 47 32.1M 47 15.2M 0 0 66.4M 0 --:--:-- --:--:-- --:--:-- 66.4M 100 32.1M 100 32.1M 0 0 104M 0 --:--:-- --:--:-- --:--:-- 219M kbcli installed successfully. Kubernetes: v1.32.6 KubeBlocks: 0.9.5 kbcli: 0.9.5-beta.8 WARNING: version difference between kbcli (0.9.5-beta.8) and kubeblocks (0.9.5) Make sure your docker service is running and begin your journey with kbcli: kbcli playground init For more information on how to get started, please visit: https://kubeblocks.io download kbcli v0.9.5-beta.8 done Kubernetes: v1.32.6 KubeBlocks: 0.9.5 kbcli: 0.9.5-beta.8 WARNING: version difference between kbcli (0.9.5-beta.8) and kubeblocks (0.9.5) Kubernetes Env: v1.32.6 POD_RESOURCES: aks kb-default-sc found aks default-vsc found found default storage class: default kubeblocks version is:0.9.5 skip upgrade kubeblocks Error: no repositories to show helm repo add chaos-mesh https://charts.chaos-mesh.org "chaos-mesh" has been added to your repositories add helm chart repo chaos-mesh success chaos mesh already installed create etcd cluster etcdo-oihqvc check component definition set component name:etcd LIMIT_CPU:0.1 LIMIT_MEMORY:0.5 storage size: 1 No resources found in ns-eicpq namespace. create 3 replica WipeOut etcd cluster check component definition set component definition by component version no component definitions found apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: name: etcdo-oihqvc namespace: ns-eicpq spec: terminationPolicy: WipeOut componentSpecs: - name: etcd componentDef: etcd serviceVersion: 3.6.1 replicas: 3 resources: requests: cpu: 100m memory: 0.5Gi limits: cpu: 100m memory: 0.5Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi services: - name: client serviceName: client spec: type: NodePort ports: - port: 2379 targetPort: 2379 componentSelector: etcd roleSelector: leader `kubectl apply -f test_create_etcdo-oihqvc.yaml` cluster.apps.kubeblocks.io/etcdo-oihqvc created apply test_create_etcdo-oihqvc.yaml Success `rm -rf test_create_etcdo-oihqvc.yaml` check cluster status `kbcli cluster list etcdo-oihqvc --show-labels --namespace ns-eicpq ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS etcdo-oihqvc ns-eicpq WipeOut Sep 01,2025 11:17 UTC+0800 cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances etcdo-oihqvc --namespace ns-eicpq ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME etcdo-oihqvc-etcd-0 ns-eicpq etcdo-oihqvc etcd Running leader 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:17 UTC+0800 etcdo-oihqvc-etcd-1 ns-eicpq etcdo-oihqvc etcd Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-default-39525487-vmss000000/10.224.0.4 Sep 01,2025 11:18 UTC+0800 etcdo-oihqvc-etcd-2 ns-eicpq etcdo-oihqvc etcd Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-68727890-vmss000007/10.224.0.5 Sep 01,2025 11:19 UTC+0800 check pod status done check cluster role check cluster role done leader: etcdo-oihqvc-etcd-0;follower: etcdo-oihqvc-etcd-1 etcdo-oihqvc-etcd-2 No resources found in ns-eicpq namespace. check cluster connect `echo 'etcdctl --endpoints=http://etcdo-oihqvc-client.ns-eicpq.svc.cluster.local:2379 endpoint health' | kubectl exec -it etcdo-oihqvc-etcd-0 --namespace ns-eicpq -- bash` check cluster connect done `kubectl get secrets -l app.kubernetes.io/instance=etcdo-oihqvc` No resources found in ns-eicpq namespace. Not found cluster secret DB_USERNAME:;DB_PASSWORD:;DB_PORT:2379;DB_DATABASE: There is no password in Type: 15. No resources found in ns-eicpq namespace. check cluster definition set component name:orioledb set component version No resources found no component version found unsupported component definition REPORT_COUNT 0:0 not found component version set replicas first:2 set replicas third:2 set minimum cmpv service version set minimum cmpv service version replicas:2 REPORT_COUNT:1 CLUSTER_TOPOLOGY: Not found topology in cluster definition orioledb LIMIT_CPU:0.1 LIMIT_MEMORY:0.5 storage size: 10 No resources found in ns-eicpq namespace. termination_policy:DoNotTerminate create 2 replica DoNotTerminate orioledb cluster check cluster version check cluster definition apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: name: orioledb-oihqvc namespace: ns-eicpq spec: clusterDefinitionRef: orioledb clusterVersionRef: orioledb-beta1 terminationPolicy: DoNotTerminate componentSpecs: - componentDefRef: orioledb name: orioledb replicas: 2 serviceRefs: - name: etcdService namespace: ns-eicpq clusterServiceSelector: cluster: etcdo-oihqvc service: component: etcd service: headless port: client resources: requests: cpu: 100m memory: 0.5Gi limits: cpu: 100m memory: 0.5Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi `kubectl apply -f test_create_orioledb-oihqvc.yaml` cluster.apps.kubeblocks.io/orioledb-oihqvc created apply test_create_orioledb-oihqvc.yaml Success `rm -rf test_create_orioledb-oihqvc.yaml` check cluster status `kbcli cluster list orioledb-oihqvc --show-labels --namespace ns-eicpq ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS orioledb-oihqvc ns-eicpq orioledb orioledb-beta1 DoNotTerminate Sep 01,2025 11:20 UTC+0800 clusterdefinition.kubeblocks.io/name=orioledb,clusterversion.kubeblocks.io/name=orioledb-beta1 cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances orioledb-oihqvc --namespace ns-eicpq ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME orioledb-oihqvc-orioledb-0 ns-eicpq orioledb-oihqvc orioledb Running primary 0 100m / 100m 512Mi / 512Mi data:10Gi aks-cicdamdpool-68727890-vmss000007/10.224.0.5 Sep 01,2025 11:20 UTC+0800 orioledb-oihqvc-orioledb-1 ns-eicpq orioledb-oihqvc orioledb Running secondary 0 100m / 100m 512Mi / 512Mi data:10Gi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:20 UTC+0800 check pod status done check cluster role check cluster role done primary: orioledb-oihqvc-orioledb-0;secondary: orioledb-oihqvc-orioledb-1 check cluster connect `echo '' | kbcli cluster connect orioledb-oihqvc --namespace ns-eicpq` check cluster connect done `kubectl get secrets -l app.kubernetes.io/instance=orioledb-oihqvc` `kubectl get secrets orioledb-oihqvc-conn-credential -o jsonpath="***.data.username***"` `kubectl get secrets orioledb-oihqvc-conn-credential -o jsonpath="***.data.password***"` `kubectl get secrets orioledb-oihqvc-conn-credential -o jsonpath="***.data.port***"` DB_USERNAME:postgres;DB_PASSWORD:pcrsvfsz;DB_PORT:5432;DB_DATABASE: check pod orioledb-oihqvc-orioledb-0 container_name postgresql exist password pcrsvfsz check pod orioledb-oihqvc-orioledb-0 container_name pgbouncer exist password pcrsvfsz Container pgbouncer logs contain secret password:+ echo '"postgres" "pcrsvfsz"' describe cluster `kbcli cluster describe orioledb-oihqvc --namespace ns-eicpq ` Name: orioledb-oihqvc Created Time: Sep 01,2025 11:20 UTC+0800 NAMESPACE CLUSTER-DEFINITION VERSION STATUS TERMINATION-POLICY ns-eicpq orioledb orioledb-beta1 Running DoNotTerminate Endpoints: COMPONENT MODE INTERNAL EXTERNAL orioledb ReadWrite orioledb-oihqvc-orioledb.ns-eicpq.svc.cluster.local:5432 orioledb-oihqvc-orioledb.ns-eicpq.svc.cluster.local:6432 Topology: COMPONENT INSTANCE ROLE STATUS AZ NODE CREATED-TIME orioledb orioledb-oihqvc-orioledb-0 primary Running 0 aks-cicdamdpool-68727890-vmss000007/10.224.0.5 Sep 01,2025 11:20 UTC+0800 orioledb orioledb-oihqvc-orioledb-1 secondary Running 0 aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:20 UTC+0800 Resources Allocation: COMPONENT DEDICATED CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS orioledb false 100m / 100m 512Mi / 512Mi data:10Gi default Images: COMPONENT TYPE IMAGE orioledb orioledb docker.io/apecloud/orioledb:beta1 Show cluster events: kbcli cluster list-events -n ns-eicpq orioledb-oihqvc `kbcli cluster label orioledb-oihqvc app.kubernetes.io/instance- --namespace ns-eicpq ` label "app.kubernetes.io/instance" not found. `kbcli cluster label orioledb-oihqvc app.kubernetes.io/instance=orioledb-oihqvc --namespace ns-eicpq ` `kbcli cluster label orioledb-oihqvc --list --namespace ns-eicpq ` NAME NAMESPACE LABELS orioledb-oihqvc ns-eicpq app.kubernetes.io/instance=orioledb-oihqvc clusterdefinition.kubeblocks.io/name=orioledb clusterversion.kubeblocks.io/name=orioledb-beta1 label cluster app.kubernetes.io/instance=orioledb-oihqvc Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=orioledb-oihqvc --namespace ns-eicpq ` `kbcli cluster label orioledb-oihqvc --list --namespace ns-eicpq ` NAME NAMESPACE LABELS orioledb-oihqvc ns-eicpq app.kubernetes.io/instance=orioledb-oihqvc case.name=kbcli.test1 clusterdefinition.kubeblocks.io/name=orioledb clusterversion.kubeblocks.io/name=orioledb-beta1 label cluster case.name=kbcli.test1 Success `kbcli cluster label orioledb-oihqvc case.name=kbcli.test2 --overwrite --namespace ns-eicpq ` `kbcli cluster label orioledb-oihqvc --list --namespace ns-eicpq ` NAME NAMESPACE LABELS orioledb-oihqvc ns-eicpq app.kubernetes.io/instance=orioledb-oihqvc case.name=kbcli.test2 clusterdefinition.kubeblocks.io/name=orioledb clusterversion.kubeblocks.io/name=orioledb-beta1 label cluster case.name=kbcli.test2 Success `kbcli cluster label orioledb-oihqvc case.name- --namespace ns-eicpq ` `kbcli cluster label orioledb-oihqvc --list --namespace ns-eicpq ` NAME NAMESPACE LABELS orioledb-oihqvc ns-eicpq app.kubernetes.io/instance=orioledb-oihqvc clusterdefinition.kubeblocks.io/name=orioledb clusterversion.kubeblocks.io/name=orioledb-beta1 delete cluster label case.name Success cluster connect `echo 'select 1' | kbcli cluster connect orioledb-oihqvc --namespace ns-eicpq ` Unable to use a TTY - input is not a terminal or the right kind of file Connect to instance orioledb-oihqvc-orioledb-0: out of orioledb-oihqvc-orioledb-0(primary), orioledb-oihqvc-orioledb-1(secondary) ?column? ---------- 1 (1 row) connect cluster Success cluster expose check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster expose orioledb-oihqvc --auto-approve --force=true --type internet --enable true --components orioledb --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-expose-6nxjp created successfully, you can view the progress: kbcli cluster describe-ops orioledb-oihqvc-expose-6nxjp -n ns-eicpq check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-expose-6nxjp ns-eicpq Expose orioledb-oihqvc Creating -/- Sep 01,2025 11:22 UTC+0800 check cluster status `kbcli cluster list orioledb-oihqvc --show-labels --namespace ns-eicpq ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS orioledb-oihqvc ns-eicpq orioledb orioledb-beta1 DoNotTerminate Running Sep 01,2025 11:20 UTC+0800 app.kubernetes.io/instance=orioledb-oihqvc,clusterdefinition.kubeblocks.io/name=orioledb,clusterversion.kubeblocks.io/name=orioledb-beta1 check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances orioledb-oihqvc --namespace ns-eicpq ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME orioledb-oihqvc-orioledb-0 ns-eicpq orioledb-oihqvc orioledb Running primary 0 100m / 100m 512Mi / 512Mi data:10Gi aks-cicdamdpool-68727890-vmss000007/10.224.0.5 Sep 01,2025 11:20 UTC+0800 orioledb-oihqvc-orioledb-1 ns-eicpq orioledb-oihqvc orioledb Running secondary 0 100m / 100m 512Mi / 512Mi data:10Gi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:20 UTC+0800 check pod status done check cluster role check cluster role done primary: orioledb-oihqvc-orioledb-0;secondary: orioledb-oihqvc-orioledb-1 check cluster connect `echo '' | kbcli cluster connect orioledb-oihqvc --namespace ns-eicpq` check cluster connect done check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-expose-6nxjp ns-eicpq Expose orioledb-oihqvc orioledb Running 0/1 Sep 01,2025 11:22 UTC+0800 ops_status:orioledb-oihqvc-expose-6nxjp ns-eicpq Expose orioledb-oihqvc orioledb Running 0/1 Sep 01,2025 11:22 UTC+0800 check ops status done ops_status:orioledb-oihqvc-expose-6nxjp ns-eicpq Expose orioledb-oihqvc orioledb Succeed 1/1 Sep 01,2025 11:22 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests orioledb-oihqvc-expose-6nxjp --namespace ns-eicpq ` opsrequest.apps.kubeblocks.io/orioledb-oihqvc-expose-6nxjp patched `kbcli cluster delete-ops --name orioledb-oihqvc-expose-6nxjp --force --auto-approve --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-expose-6nxjp deleted cluster vscale check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale orioledb-oihqvc --auto-approve --force=true --components orioledb --cpu 200m --memory 0.6Gi --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-verticalscaling-ft4wn created successfully, you can view the progress: kbcli cluster describe-ops orioledb-oihqvc-verticalscaling-ft4wn -n ns-eicpq check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-verticalscaling-ft4wn ns-eicpq VerticalScaling orioledb-oihqvc orioledb Running -/- Sep 01,2025 11:22 UTC+0800 check cluster status `kbcli cluster list orioledb-oihqvc --show-labels --namespace ns-eicpq ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS orioledb-oihqvc ns-eicpq orioledb orioledb-beta1 DoNotTerminate Updating Sep 01,2025 11:20 UTC+0800 app.kubernetes.io/instance=orioledb-oihqvc,clusterdefinition.kubeblocks.io/name=orioledb,clusterversion.kubeblocks.io/name=orioledb-beta1 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances orioledb-oihqvc --namespace ns-eicpq ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME orioledb-oihqvc-orioledb-0 ns-eicpq orioledb-oihqvc orioledb Running primary 0 200m / 200m 644245094400m / 644245094400m data:10Gi aks-cicdamdpool-68727890-vmss000007/10.224.0.5 Sep 01,2025 11:24 UTC+0800 orioledb-oihqvc-orioledb-1 ns-eicpq orioledb-oihqvc orioledb Running secondary 0 200m / 200m 644245094400m / 644245094400m data:10Gi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:23 UTC+0800 check pod status done check cluster role check cluster role done primary: orioledb-oihqvc-orioledb-0;secondary: orioledb-oihqvc-orioledb-1 check cluster connect `echo '' | kbcli cluster connect orioledb-oihqvc --namespace ns-eicpq` check cluster connect done check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-verticalscaling-ft4wn ns-eicpq VerticalScaling orioledb-oihqvc orioledb Succeed 2/2 Sep 01,2025 11:22 UTC+0800 check ops status done ops_status:orioledb-oihqvc-verticalscaling-ft4wn ns-eicpq VerticalScaling orioledb-oihqvc orioledb Succeed 2/2 Sep 01,2025 11:22 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests orioledb-oihqvc-verticalscaling-ft4wn --namespace ns-eicpq ` opsrequest.apps.kubeblocks.io/orioledb-oihqvc-verticalscaling-ft4wn patched `kbcli cluster delete-ops --name orioledb-oihqvc-verticalscaling-ft4wn --force --auto-approve --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-verticalscaling-ft4wn deleted skip Reconfiguring: cluster hscale check cluster status before ops check cluster status done cluster_status:Running No resources found in orioledb-oihqvc namespace. `kbcli cluster hscale orioledb-oihqvc --auto-approve --force=true --components orioledb --replicas 3 --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-horizontalscaling-zxjrs created successfully, you can view the progress: kbcli cluster describe-ops orioledb-oihqvc-horizontalscaling-zxjrs -n ns-eicpq check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-horizontalscaling-zxjrs ns-eicpq HorizontalScaling orioledb-oihqvc orioledb Creating -/- Sep 01,2025 11:25 UTC+0800 check cluster status `kbcli cluster list orioledb-oihqvc --show-labels --namespace ns-eicpq ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS orioledb-oihqvc ns-eicpq orioledb orioledb-beta1 DoNotTerminate Updating Sep 01,2025 11:20 UTC+0800 app.kubernetes.io/instance=orioledb-oihqvc,clusterdefinition.kubeblocks.io/name=orioledb,clusterversion.kubeblocks.io/name=orioledb-beta1 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances orioledb-oihqvc --namespace ns-eicpq ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME orioledb-oihqvc-orioledb-0 ns-eicpq orioledb-oihqvc orioledb Running primary 0 200m / 200m 644245094400m / 644245094400m data:10Gi aks-cicdamdpool-68727890-vmss000007/10.224.0.5 Sep 01,2025 11:24 UTC+0800 orioledb-oihqvc-orioledb-1 ns-eicpq orioledb-oihqvc orioledb Running secondary 0 200m / 200m 644245094400m / 644245094400m data:10Gi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:23 UTC+0800 orioledb-oihqvc-orioledb-2 ns-eicpq orioledb-oihqvc orioledb Running secondary 0 200m / 200m 644245094400m / 644245094400m data:10Gi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:25 UTC+0800 check pod status done check cluster role check cluster role done primary: orioledb-oihqvc-orioledb-0;secondary: orioledb-oihqvc-orioledb-1 orioledb-oihqvc-orioledb-2 check cluster connect `echo '' | kbcli cluster connect orioledb-oihqvc --namespace ns-eicpq` check cluster connect done No resources found in orioledb-oihqvc namespace. check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-horizontalscaling-zxjrs ns-eicpq HorizontalScaling orioledb-oihqvc orioledb Succeed 1/1 Sep 01,2025 11:25 UTC+0800 check ops status done ops_status:orioledb-oihqvc-horizontalscaling-zxjrs ns-eicpq HorizontalScaling orioledb-oihqvc orioledb Succeed 1/1 Sep 01,2025 11:25 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests orioledb-oihqvc-horizontalscaling-zxjrs --namespace ns-eicpq ` opsrequest.apps.kubeblocks.io/orioledb-oihqvc-horizontalscaling-zxjrs patched `kbcli cluster delete-ops --name orioledb-oihqvc-horizontalscaling-zxjrs --force --auto-approve --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-horizontalscaling-zxjrs deleted cluster hscale check cluster status before ops check cluster status done cluster_status:Running No resources found in orioledb-oihqvc namespace. `kbcli cluster hscale orioledb-oihqvc --auto-approve --force=true --components orioledb --replicas 2 --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-horizontalscaling-xvc4k created successfully, you can view the progress: kbcli cluster describe-ops orioledb-oihqvc-horizontalscaling-xvc4k -n ns-eicpq check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-horizontalscaling-xvc4k ns-eicpq HorizontalScaling orioledb-oihqvc orioledb Pending -/- Sep 01,2025 11:26 UTC+0800 check cluster status `kbcli cluster list orioledb-oihqvc --show-labels --namespace ns-eicpq ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS orioledb-oihqvc ns-eicpq orioledb orioledb-beta1 DoNotTerminate Updating Sep 01,2025 11:20 UTC+0800 app.kubernetes.io/instance=orioledb-oihqvc,clusterdefinition.kubeblocks.io/name=orioledb,clusterversion.kubeblocks.io/name=orioledb-beta1 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances orioledb-oihqvc --namespace ns-eicpq ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME orioledb-oihqvc-orioledb-0 ns-eicpq orioledb-oihqvc orioledb Running primary 0 200m / 200m 644245094400m / 644245094400m data:10Gi aks-cicdamdpool-68727890-vmss000007/10.224.0.5 Sep 01,2025 11:24 UTC+0800 orioledb-oihqvc-orioledb-1 ns-eicpq orioledb-oihqvc orioledb Running secondary 0 200m / 200m 644245094400m / 644245094400m data:10Gi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:23 UTC+0800 check pod status done check cluster role check cluster role done primary: orioledb-oihqvc-orioledb-0;secondary: orioledb-oihqvc-orioledb-1 check cluster connect `echo '' | kbcli cluster connect orioledb-oihqvc --namespace ns-eicpq` check cluster connect done No resources found in orioledb-oihqvc namespace. check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-horizontalscaling-xvc4k ns-eicpq HorizontalScaling orioledb-oihqvc orioledb Succeed 1/1 Sep 01,2025 11:26 UTC+0800 check ops status done ops_status:orioledb-oihqvc-horizontalscaling-xvc4k ns-eicpq HorizontalScaling orioledb-oihqvc orioledb Succeed 1/1 Sep 01,2025 11:26 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests orioledb-oihqvc-horizontalscaling-xvc4k --namespace ns-eicpq ` opsrequest.apps.kubeblocks.io/orioledb-oihqvc-horizontalscaling-xvc4k patched `kbcli cluster delete-ops --name orioledb-oihqvc-horizontalscaling-xvc4k --force --auto-approve --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-horizontalscaling-xvc4k deleted cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart orioledb-oihqvc --auto-approve --force=true --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-restart-76cl9 created successfully, you can view the progress: kbcli cluster describe-ops orioledb-oihqvc-restart-76cl9 -n ns-eicpq check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Creating -/- Sep 01,2025 11:27 UTC+0800 check cluster status `kbcli cluster list orioledb-oihqvc --show-labels --namespace ns-eicpq ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS orioledb-oihqvc ns-eicpq orioledb orioledb-beta1 DoNotTerminate Updating Sep 01,2025 11:20 UTC+0800 app.kubernetes.io/instance=orioledb-oihqvc,clusterdefinition.kubeblocks.io/name=orioledb,clusterversion.kubeblocks.io/name=orioledb-beta1 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating [Error] check cluster status timeout --------------------------------------get cluster orioledb-oihqvc yaml-------------------------------------- `kubectl get cluster orioledb-oihqvc -o yaml --namespace ns-eicpq ` apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: annotations: kubeblocks.io/ops-request: '[***"name":"orioledb-oihqvc-restart-76cl9","type":"Restart"***]' kubeblocks.io/reconcile: "2025-09-01T03:34:48.20060711Z" kubectl.kubernetes.io/last-applied-configuration: | ***"apiVersion":"apps.kubeblocks.io/v1alpha1","kind":"Cluster","metadata":***"annotations":***,"name":"orioledb-oihqvc","namespace":"ns-eicpq"***,"spec":***"clusterDefinitionRef":"orioledb","clusterVersionRef":"orioledb-beta1","componentSpecs":[***"componentDefRef":"orioledb","name":"orioledb","replicas":2,"resources":***"limits":***"cpu":"100m","memory":"0.5Gi"***,"requests":***"cpu":"100m","memory":"0.5Gi"***,"serviceRefs":[***"clusterServiceSelector":***"cluster":"etcdo-oihqvc","service":***"component":"etcd","port":"client","service":"headless"***,"name":"etcdService","namespace":"ns-eicpq"***],"volumeClaimTemplates":[***"name":"data","spec":***"accessModes":["ReadWriteOnce"],"resources":***"requests":***"storage":"10Gi"***,"storageClassName":null***]***],"terminationPolicy":"DoNotTerminate"*** creationTimestamp: "2025-09-01T03:20:35Z" finalizers: - cluster.kubeblocks.io/finalizer generation: 7 labels: app.kubernetes.io/instance: orioledb-oihqvc clusterdefinition.kubeblocks.io/name: orioledb clusterversion.kubeblocks.io/name: orioledb-beta1 name: orioledb-oihqvc namespace: ns-eicpq resourceVersion: "31257" uid: 26a1e640-3089-414c-bb4c-1f68b1c5af12 spec: clusterDefinitionRef: orioledb clusterVersionRef: orioledb-beta1 componentSpecs: - componentDefRef: orioledb name: orioledb replicas: 2 resources: limits: cpu: 200m memory: 644245094400m requests: cpu: 200m memory: 644245094400m serviceRefs: - clusterServiceSelector: cluster: etcdo-oihqvc service: component: etcd port: client service: headless name: etcdService namespace: ns-eicpq volumeClaimTemplates: - name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi resources: cpu: "0" memory: "0" services: - annotations: service.beta.kubernetes.io/azure-load-balancer-internal: "false" componentSelector: orioledb name: orioledb-internet roleSelector: primary serviceName: orioledb-internet spec: ports: - name: tcp-orioledb nodePort: 30490 port: 5432 protocol: TCP targetPort: tcp-orioledb - name: tcp-pgbouncer nodePort: 32393 port: 6432 protocol: TCP targetPort: tcp-pgbouncer type: LoadBalancer storage: size: "0" terminationPolicy: DoNotTerminate status: clusterDefGeneration: 2 components: orioledb: phase: Updating podsReady: false podsReadyTime: "2025-09-01T03:27:31Z" conditions: - lastTransitionTime: "2025-09-01T03:20:35Z" message: 'The operator has started the provisioning of Cluster: orioledb-oihqvc' observedGeneration: 7 reason: PreCheckSucceed status: "True" type: ProvisioningStarted - lastTransitionTime: "2025-09-01T03:20:35Z" message: Successfully applied for resources observedGeneration: 7 reason: ApplyResourcesSucceed status: "True" type: ApplyResources - lastTransitionTime: "2025-09-01T03:27:39Z" message: 'pods are not ready in Components: [orioledb], refer to related component message in Cluster.status.components' reason: ReplicasNotReady status: "False" type: ReplicasReady - lastTransitionTime: "2025-09-01T03:27:39Z" message: 'pods are unavailable in Components: [orioledb], refer to related component message in Cluster.status.components' reason: ComponentsNotReady status: "False" type: Ready observedGeneration: 7 phase: Updating ------------------------------------------------------------------------------------------------------------------ --------------------------------------describe cluster orioledb-oihqvc-------------------------------------- `kubectl describe cluster orioledb-oihqvc --namespace ns-eicpq ` Name: orioledb-oihqvc Namespace: ns-eicpq Labels: app.kubernetes.io/instance=orioledb-oihqvc clusterdefinition.kubeblocks.io/name=orioledb clusterversion.kubeblocks.io/name=orioledb-beta1 Annotations: kubeblocks.io/ops-request: [***"name":"orioledb-oihqvc-restart-76cl9","type":"Restart"***] kubeblocks.io/reconcile: 2025-09-01T03:34:48.20060711Z API Version: apps.kubeblocks.io/v1alpha1 Kind: Cluster Metadata: Creation Timestamp: 2025-09-01T03:20:35Z Finalizers: cluster.kubeblocks.io/finalizer Generation: 7 Resource Version: 31257 UID: 26a1e640-3089-414c-bb4c-1f68b1c5af12 Spec: Cluster Definition Ref: orioledb Cluster Version Ref: orioledb-beta1 Component Specs: Component Def Ref: orioledb Name: orioledb Replicas: 2 Resources: Limits: Cpu: 200m Memory: 644245094400m Requests: Cpu: 200m Memory: 644245094400m Service Refs: Cluster Service Selector: Cluster: etcdo-oihqvc Service: Component: etcd Port: client Service: headless Name: etcdService Namespace: ns-eicpq Volume Claim Templates: Name: data Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 10Gi Resources: Cpu: 0 Memory: 0 Services: Annotations: service.beta.kubernetes.io/azure-load-balancer-internal: false Component Selector: orioledb Name: orioledb-internet Role Selector: primary Service Name: orioledb-internet Spec: Ports: Name: tcp-orioledb Node Port: 30490 Port: 5432 Protocol: TCP Target Port: tcp-orioledb Name: tcp-pgbouncer Node Port: 32393 Port: 6432 Protocol: TCP Target Port: tcp-pgbouncer Type: LoadBalancer Storage: Size: 0 Termination Policy: DoNotTerminate Status: Cluster Def Generation: 2 Components: Orioledb: Phase: Updating Pods Ready: false Pods Ready Time: 2025-09-01T03:27:31Z Conditions: Last Transition Time: 2025-09-01T03:20:35Z Message: The operator has started the provisioning of Cluster: orioledb-oihqvc Observed Generation: 7 Reason: PreCheckSucceed Status: True Type: ProvisioningStarted Last Transition Time: 2025-09-01T03:20:35Z Message: Successfully applied for resources Observed Generation: 7 Reason: ApplyResourcesSucceed Status: True Type: ApplyResources Last Transition Time: 2025-09-01T03:27:39Z Message: pods are not ready in Components: [orioledb], refer to related component message in Cluster.status.components Reason: ReplicasNotReady Status: False Type: ReplicasReady Last Transition Time: 2025-09-01T03:27:39Z Message: pods are unavailable in Components: [orioledb], refer to related component message in Cluster.status.components Reason: ComponentsNotReady Status: False Type: Ready Observed Generation: 7 Phase: Updating Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ComponentPhaseTransition 14m cluster-controller component is Creating Warning Unhealthy 13m (x2 over 13m) event-controller Pod orioledb-oihqvc-orioledb-1: Readiness probe failed: 127.0.0.1:5432 - no response Warning ReplicasNotReady 12m cluster-controller pods are not ready in Components: [orioledb], refer to related component message in Cluster.status.components Warning ComponentsNotReady 12m cluster-controller pods are unavailable in Components: [orioledb], refer to related component message in Cluster.status.components Normal AllReplicasReady 9m44s (x2 over 13m) cluster-controller all pods of components are ready, waiting for the probe detection successful Normal ClusterReady 9m44s (x2 over 13m) cluster-controller Cluster: orioledb-oihqvc is ready, current phase is Running Normal Running 9m44s (x2 over 13m) cluster-controller Cluster: orioledb-oihqvc is ready, current phase is Running Normal ApplyResourcesSucceed 9m39s (x6 over 14m) cluster-controller Successfully applied for resources Normal PreCheckSucceed 9m39s (x6 over 14m) cluster-controller The operator has started the provisioning of Cluster: orioledb-oihqvc Normal HorizontalScale 9m38s (x2 over 9m38s) component-controller start horizontal scale component orioledb of cluster orioledb-oihqvc from 2 to 3 Normal ComponentPhaseTransition 9m37s (x2 over 12m) cluster-controller component is Updating Normal ComponentPhaseTransition 8m18s (x3 over 13m) cluster-controller component is Running Normal HorizontalScale 8m11s component-controller start horizontal scale component orioledb of cluster orioledb-oihqvc from 3 to 2 Warning FailedAttachVolume 22s event-controller Pod orioledb-oihqvc-orioledb-1: AttachVolume.Attach failed for volume "pvc-312894ce-2ebb-4f02-a8bf-1fe4e338a71b" : timed out waiting for external-attacher of disk.csi.azure.com CSI driver to attach volume /subscriptions/e659c16c-8ba9-41ab-98a0-d64b0237ba45/resourceGroups/MC_cicd-aks-qroep9l1-group_cicd-aks-qroep9l1_eastus/providers/Microsoft.Compute/disks/pvc-312894ce-2ebb-4f02-a8bf-1fe4e338a71b ------------------------------------------------------------------------------------------------------------------ check pod status `kbcli cluster list-instances orioledb-oihqvc --namespace ns-eicpq ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME orioledb-oihqvc-orioledb-0 ns-eicpq orioledb-oihqvc orioledb Running primary 0 200m / 200m 644245094400m / 644245094400m data:10Gi aks-cicdamdpool-68727890-vmss000007/10.224.0.5 Sep 01,2025 11:24 UTC+0800 orioledb-oihqvc-orioledb-1 ns-eicpq orioledb-oihqvc orioledb ContainerCreating 0 200m / 200m 644245094400m / 644245094400m data:10Gi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:28 UTC+0800 pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating pod_status:ContainerCreating check pod status done check cluster status again check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 0/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 0/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 0/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 0/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 0/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 0/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 0/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 0/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 0/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Running 1/2 Sep 01,2025 11:27 UTC+0800 check ops status done ops_status:orioledb-oihqvc-restart-76cl9 ns-eicpq Restart orioledb-oihqvc orioledb Succeed 2/2 Sep 01,2025 11:27 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests orioledb-oihqvc-restart-76cl9 --namespace ns-eicpq ` opsrequest.apps.kubeblocks.io/orioledb-oihqvc-restart-76cl9 patched `kbcli cluster delete-ops --name orioledb-oihqvc-restart-76cl9 --force --auto-approve --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-restart-76cl9 deleted `kubectl get pvc -l app.kubernetes.io/instance=orioledb-oihqvc,apps.kubeblocks.io/component-name=orioledb,apps.kubeblocks.io/vct-name=data --namespace ns-eicpq ` cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in orioledb-oihqvc namespace. `kbcli cluster volume-expand orioledb-oihqvc --auto-approve --force=true --components orioledb --volume-claim-templates data --storage 15Gi --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-volumeexpansion-tgcnh created successfully, you can view the progress: kbcli cluster describe-ops orioledb-oihqvc-volumeexpansion-tgcnh -n ns-eicpq check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-volumeexpansion-tgcnh ns-eicpq VolumeExpansion orioledb-oihqvc orioledb Running -/- Sep 01,2025 11:40 UTC+0800 check cluster status `kbcli cluster list orioledb-oihqvc --show-labels --namespace ns-eicpq ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS orioledb-oihqvc ns-eicpq orioledb orioledb-beta1 DoNotTerminate Updating Sep 01,2025 11:20 UTC+0800 app.kubernetes.io/instance=orioledb-oihqvc,clusterdefinition.kubeblocks.io/name=orioledb,clusterversion.kubeblocks.io/name=orioledb-beta1 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances orioledb-oihqvc --namespace ns-eicpq ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME orioledb-oihqvc-orioledb-0 ns-eicpq orioledb-oihqvc orioledb Running secondary 0 200m / 200m 644245094400m / 644245094400m data:15Gi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:39 UTC+0800 orioledb-oihqvc-orioledb-1 ns-eicpq orioledb-oihqvc orioledb Running primary 0 200m / 200m 644245094400m / 644245094400m data:15Gi aks-cicdamdpool-68727890-vmss000001/10.224.0.7 Sep 01,2025 11:28 UTC+0800 check pod status done check cluster role check cluster role done primary: orioledb-oihqvc-orioledb-1;secondary: orioledb-oihqvc-orioledb-0 check cluster connect `echo '' | kbcli cluster connect orioledb-oihqvc --namespace ns-eicpq` check cluster connect done No resources found in orioledb-oihqvc namespace. check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-volumeexpansion-tgcnh ns-eicpq VolumeExpansion orioledb-oihqvc orioledb Succeed 2/2 Sep 01,2025 11:40 UTC+0800 check ops status done ops_status:orioledb-oihqvc-volumeexpansion-tgcnh ns-eicpq VolumeExpansion orioledb-oihqvc orioledb Succeed 2/2 Sep 01,2025 11:40 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests orioledb-oihqvc-volumeexpansion-tgcnh --namespace ns-eicpq ` opsrequest.apps.kubeblocks.io/orioledb-oihqvc-volumeexpansion-tgcnh patched `kbcli cluster delete-ops --name orioledb-oihqvc-volumeexpansion-tgcnh --force --auto-approve --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-volumeexpansion-tgcnh deleted cluster stop check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster stop orioledb-oihqvc --auto-approve --force=true --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-stop-bgntq created successfully, you can view the progress: kbcli cluster describe-ops orioledb-oihqvc-stop-bgntq -n ns-eicpq check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-stop-bgntq ns-eicpq Stop orioledb-oihqvc Running -/- Sep 01,2025 11:43 UTC+0800 check cluster status `kbcli cluster list orioledb-oihqvc --show-labels --namespace ns-eicpq ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS orioledb-oihqvc ns-eicpq orioledb orioledb-beta1 DoNotTerminate Stopping Sep 01,2025 11:20 UTC+0800 app.kubernetes.io/instance=orioledb-oihqvc,clusterdefinition.kubeblocks.io/name=orioledb,clusterversion.kubeblocks.io/name=orioledb-beta1 cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping check cluster status done cluster_status:Stopped check pod status `kbcli cluster list-instances orioledb-oihqvc --namespace ns-eicpq ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME check pod status done check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-stop-bgntq ns-eicpq Stop orioledb-oihqvc orioledb Succeed 2/2 Sep 01,2025 11:43 UTC+0800 check ops status done ops_status:orioledb-oihqvc-stop-bgntq ns-eicpq Stop orioledb-oihqvc orioledb Succeed 2/2 Sep 01,2025 11:43 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests orioledb-oihqvc-stop-bgntq --namespace ns-eicpq ` opsrequest.apps.kubeblocks.io/orioledb-oihqvc-stop-bgntq patched `kbcli cluster delete-ops --name orioledb-oihqvc-stop-bgntq --force --auto-approve --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-stop-bgntq deleted cluster start check cluster status before ops check cluster status done cluster_status:Stopped `kbcli cluster start orioledb-oihqvc --force=true --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-start-ww8lf created successfully, you can view the progress: kbcli cluster describe-ops orioledb-oihqvc-start-ww8lf -n ns-eicpq check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-start-ww8lf ns-eicpq Start orioledb-oihqvc Running -/- Sep 01,2025 11:44 UTC+0800 check cluster status `kbcli cluster list orioledb-oihqvc --show-labels --namespace ns-eicpq ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS orioledb-oihqvc ns-eicpq orioledb orioledb-beta1 DoNotTerminate Updating Sep 01,2025 11:20 UTC+0800 app.kubernetes.io/instance=orioledb-oihqvc,clusterdefinition.kubeblocks.io/name=orioledb,clusterversion.kubeblocks.io/name=orioledb-beta1 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances orioledb-oihqvc --namespace ns-eicpq ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME orioledb-oihqvc-orioledb-0 ns-eicpq orioledb-oihqvc orioledb Running primary 0 200m / 200m 644245094400m / 644245094400m data:15Gi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:44 UTC+0800 orioledb-oihqvc-orioledb-1 ns-eicpq orioledb-oihqvc orioledb Running secondary 0 200m / 200m 644245094400m / 644245094400m data:15Gi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:44 UTC+0800 check pod status done check cluster role check cluster role done primary: orioledb-oihqvc-orioledb-0;secondary: orioledb-oihqvc-orioledb-1 check cluster connect `echo '' | kbcli cluster connect orioledb-oihqvc --namespace ns-eicpq` check cluster connect done check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-start-ww8lf ns-eicpq Start orioledb-oihqvc orioledb Succeed 2/2 Sep 01,2025 11:44 UTC+0800 check ops status done ops_status:orioledb-oihqvc-start-ww8lf ns-eicpq Start orioledb-oihqvc orioledb Succeed 2/2 Sep 01,2025 11:44 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests orioledb-oihqvc-start-ww8lf --namespace ns-eicpq ` opsrequest.apps.kubeblocks.io/orioledb-oihqvc-start-ww8lf patched `kbcli cluster delete-ops --name orioledb-oihqvc-start-ww8lf --force --auto-approve --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-start-ww8lf deleted `kubectl get secrets -l app.kubernetes.io/instance=orioledb-oihqvc` `kubectl get secrets orioledb-oihqvc-conn-credential -o jsonpath="***.data.username***"` `kubectl get secrets orioledb-oihqvc-conn-credential -o jsonpath="***.data.password***"` `kubectl get secrets orioledb-oihqvc-conn-credential -o jsonpath="***.data.port***"` DB_USERNAME:postgres;DB_PASSWORD:pcrsvfsz;DB_PORT:5432;DB_DATABASE: `create database benchtest;` Unable to use a TTY - input is not a terminal or the right kind of file NOTICE: database "benchtest" does not exist, skipping return msg:Connect to instance orioledb-oihqvc-orioledb-0: out of orioledb-oihqvc-orioledb-0(primary), orioledb-oihqvc-orioledb-1(secondary) DROP DATABASE CREATE DATABASE apiVersion: v1 kind: Pod metadata: name: benchtest-orioledb-oihqvc namespace: ns-eicpq spec: containers: - name: test-sysbench imagePullPolicy: IfNotPresent image: docker.io/apecloud/customsuites:latest env: - name: TYPE value: "2" - name: FLAG value: "0" - name: CONFIGS value: "mode:all,driver:pgsql,host:orioledb-oihqvc-orioledb.ns-eicpq.svc.cluster.local,user:postgres,password:pcrsvfsz,port:5432,db:benchtest,tables:5,threads:4,times:10,size:1000,type:oltp_read_write" restartPolicy: Never `kubectl apply -f benchtest-orioledb-oihqvc.yaml` pod/benchtest-orioledb-oihqvc created apply benchtest-orioledb-oihqvc.yaml Success check pod status pod_status:NAME READY STATUS RESTARTS AGE benchtest-orioledb-oihqvc 0/1 ContainerCreating 0 1s pod_status:NAME READY STATUS RESTARTS AGE benchtest-orioledb-oihqvc 0/1 ContainerCreating 0 5s pod_status:NAME READY STATUS RESTARTS AGE benchtest-orioledb-oihqvc 0/1 ContainerCreating 0 10s pod_status:NAME READY STATUS RESTARTS AGE benchtest-orioledb-oihqvc 0/1 ContainerCreating 0 15s pod_status:NAME READY STATUS RESTARTS AGE benchtest-orioledb-oihqvc 0/1 ContainerCreating 0 20s pod_status:NAME READY STATUS RESTARTS AGE benchtest-orioledb-oihqvc 1/1 Running 0 26s pod_status:NAME READY STATUS RESTARTS AGE benchtest-orioledb-oihqvc 1/1 Running 0 31s check pod benchtest-orioledb-oihqvc status done pod_status:NAME READY STATUS RESTARTS AGE benchtest-orioledb-oihqvc 0/1 Completed 0 36s `rm -rf benchtest-orioledb-oihqvc.yaml` `kubectl logs benchtest-orioledb-oihqvc --tail 30 --namespace ns-eicpq ` [ 6s ] thds: 4 tps: 79.00 qps: 1618.02 (r/w/o: 1135.01/323.00/160.00) lat (ms,99%): 99.33 err/s: 1.00 reconn/s: 0.00 [ 7s ] thds: 4 tps: 80.99 qps: 1628.81 (r/w/o: 1143.86/318.96/165.98) lat (ms,99%): 99.33 err/s: 1.00 reconn/s: 0.00 [ 8s ] thds: 4 tps: 87.01 qps: 1729.18 (r/w/o: 1209.13/345.04/175.02) lat (ms,99%): 97.55 err/s: 0.00 reconn/s: 0.00 [ 9s ] thds: 4 tps: 81.99 qps: 1651.88 (r/w/o: 1159.92/327.98/163.99) lat (ms,99%): 97.55 err/s: 0.00 reconn/s: 0.00 SQL statistics: queries performed: read: 11340 write: 3224 other: 1626 total: 16190 transactions: 805 (80.46 per sec.) queries: 16190 (1618.26 per sec.) ignored errors: 5 (0.50 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 10.0026s total number of events: 805 Latency (ms): min: 2.32 avg: 49.69 max: 102.76 99th percentile: 101.13 sum: 40001.02 Threads fairness: events (avg/stddev): 201.2500/3.11 execution time (avg/stddev): 10.0003/0.00 `kubectl delete pod benchtest-orioledb-oihqvc --force --namespace ns-eicpq ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "benchtest-orioledb-oihqvc" force deleted No resources found in ns-eicpq namespace. `kubectl get secrets -l app.kubernetes.io/instance=orioledb-oihqvc` `kubectl get secrets orioledb-oihqvc-conn-credential -o jsonpath="***.data.username***"` `kubectl get secrets orioledb-oihqvc-conn-credential -o jsonpath="***.data.password***"` `kubectl get secrets orioledb-oihqvc-conn-credential -o jsonpath="***.data.port***"` DB_USERNAME:postgres;DB_PASSWORD:pcrsvfsz;DB_PORT:5432;DB_DATABASE: `create database benchtest;` Unable to use a TTY - input is not a terminal or the right kind of file return msg:Connect to instance orioledb-oihqvc-orioledb-0: out of orioledb-oihqvc-orioledb-0(primary), orioledb-oihqvc-orioledb-1(secondary) DROP DATABASE CREATE DATABASE apiVersion: v1 kind: Pod metadata: name: benchtest-orioledb-oihqvc namespace: ns-eicpq spec: containers: - name: test-sysbench imagePullPolicy: IfNotPresent image: docker.io/apecloud/customsuites:latest env: - name: TYPE value: "2" - name: FLAG value: "0" - name: CONFIGS value: "mode:all,driver:pgsql,host:4.156.49.69,user:postgres,password:pcrsvfsz,port:5432,db:benchtest,tables:5,threads:4,times:10,size:1000,type:oltp_read_write" restartPolicy: Never `kubectl apply -f benchtest-orioledb-oihqvc.yaml` pod/benchtest-orioledb-oihqvc created apply benchtest-orioledb-oihqvc.yaml Success check pod status pod_status:NAME READY STATUS RESTARTS AGE benchtest-orioledb-oihqvc 0/1 ContainerCreating 0 1s pod_status:NAME READY STATUS RESTARTS AGE benchtest-orioledb-oihqvc 1/1 Running 0 5s pod_status:NAME READY STATUS RESTARTS AGE benchtest-orioledb-oihqvc 1/1 Running 0 10s check pod benchtest-orioledb-oihqvc status done pod_status:NAME READY STATUS RESTARTS AGE benchtest-orioledb-oihqvc 0/1 Completed 0 15s `rm -rf benchtest-orioledb-oihqvc.yaml` `kubectl logs benchtest-orioledb-oihqvc --tail 30 --namespace ns-eicpq ` [ 7s ] thds: 4 tps: 90.01 qps: 1789.17 (r/w/o: 1250.12/359.03/180.02) lat (ms,99%): 97.55 err/s: 0.00 reconn/s: 0.00 [ 8s ] thds: 4 tps: 89.00 qps: 1817.92 (r/w/o: 1277.95/357.99/181.99) lat (ms,99%): 97.55 err/s: 1.00 reconn/s: 0.00 [ 9s ] thds: 4 tps: 39.00 qps: 789.04 (r/w/o: 551.03/159.01/79.00) lat (ms,99%): 101.13 err/s: 0.00 reconn/s: 0.00 [ 10s ] thds: 4 tps: 5.00 qps: 116.00 (r/w/o: 84.00/21.00/11.00) lat (ms,99%): 1109.09 err/s: 1.00 reconn/s: 0.00 SQL statistics: queries performed: read: 8988 write: 2556 other: 1287 total: 12831 transactions: 639 (60.28 per sec.) queries: 12831 (1210.43 per sec.) ignored errors: 3 (0.28 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 10.5983s total number of events: 639 Latency (ms): min: 2.34 avg: 66.33 max: 2101.08 99th percentile: 101.13 sum: 42381.75 Threads fairness: events (avg/stddev): 159.7500/4.92 execution time (avg/stddev): 10.5954/0.00 `kubectl delete pod benchtest-orioledb-oihqvc --force --namespace ns-eicpq ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "benchtest-orioledb-oihqvc" force deleted skip Reconfiguring: cluster update terminationPolicy WipeOut `kbcli cluster update orioledb-oihqvc --termination-policy=WipeOut --namespace ns-eicpq ` cluster.apps.kubeblocks.io/orioledb-oihqvc updated check cluster status `kbcli cluster list orioledb-oihqvc --show-labels --namespace ns-eicpq ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS orioledb-oihqvc ns-eicpq orioledb orioledb-beta1 WipeOut Running Sep 01,2025 11:20 UTC+0800 app.kubernetes.io/instance=orioledb-oihqvc,clusterdefinition.kubeblocks.io/name=orioledb,clusterversion.kubeblocks.io/name=orioledb-beta1 check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances orioledb-oihqvc --namespace ns-eicpq ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME orioledb-oihqvc-orioledb-0 ns-eicpq orioledb-oihqvc orioledb Running primary 0 200m / 200m 644245094400m / 644245094400m data:15Gi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:44 UTC+0800 orioledb-oihqvc-orioledb-1 ns-eicpq orioledb-oihqvc orioledb Running secondary 0 200m / 200m 644245094400m / 644245094400m data:15Gi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:44 UTC+0800 check pod status done check cluster role check cluster role done primary: orioledb-oihqvc-orioledb-0;secondary: orioledb-oihqvc-orioledb-1 check cluster connect `echo '' | kbcli cluster connect orioledb-oihqvc --namespace ns-eicpq` check cluster connect done cluster list-logs `kbcli cluster list-logs orioledb-oihqvc --namespace ns-eicpq ` No log files found. You can enable the log feature with the kbcli command below. kbcli cluster update orioledb-oihqvc --enable-all-logs=true --namespace ns-eicpq Error from server (NotFound): pods "orioledb-oihqvc-orioledb-0" not found cluster logs `kbcli cluster logs orioledb-oihqvc --tail 30 --namespace ns-eicpq ` Defaulted container "postgresql" out of: postgresql, pgbouncer, metrics, lorry, config-manager PRIMARY KEY (id) ) 2025-09-01 03:52:29.559 GMT [241] LOG: statement: CREATE INDEX k_5 ON sbtest5(k) 2025-09-01 03:52:34,054 INFO: no action. I am (orioledb-oihqvc-orioledb-0), the leader with the lock 2025-09-01 03:52:37.756 GMT [250] ERROR: duplicate key value violates unique constraint "sbtest3_pkey" 2025-09-01 03:52:37.756 GMT [250] DETAIL: Key (id)=(503) already exists. 2025-09-01 03:52:37.756 GMT [250] STATEMENT: INSERT INTO sbtest3 (id, k, c, pad) VALUES ($1, $2, $3, $4) 2025-09-01 03:52:39.354 GMT [251] ERROR: deadlock detected 2025-09-01 03:52:39.354 GMT [251] DETAIL: Process 251 waits for ShareLock on transaction 2235; blocked by process 252. Process 252 waits for ShareLock on transaction 2236; blocked by process 250. Process 250 waits for AccessExclusiveLock on tuple (27,158) of relation 24690 of database 24677; blocked by process 251. Process 251: UPDATE sbtest3 SET c=$1 WHERE id=$2 Process 252: UPDATE sbtest3 SET c=$1 WHERE id=$2 Process 250: DELETE FROM sbtest3 WHERE id=$1 2025-09-01 03:52:39.354 GMT [251] HINT: See server log for query details. 2025-09-01 03:52:39.354 GMT [251] CONTEXT: while updating tuple (27,158) in relation "sbtest3" 2025-09-01 03:52:39.354 GMT [251] STATEMENT: UPDATE sbtest3 SET c=$1 WHERE id=$2 2025-09-01 03:52:40.354 GMT [250] ERROR: deadlock detected 2025-09-01 03:52:40.354 GMT [250] DETAIL: Process 250 waits for ShareLock on transaction 2235; blocked by process 252. Process 252 waits for ShareLock on transaction 2236; blocked by process 250. Process 250: DELETE FROM sbtest3 WHERE id=$1 Process 252: UPDATE sbtest3 SET c=$1 WHERE id=$2 2025-09-01 03:52:40.354 GMT [250] HINT: See server log for query details. 2025-09-01 03:52:40.354 GMT [250] CONTEXT: while deleting tuple (27,158) in relation "sbtest3" 2025-09-01 03:52:40.354 GMT [250] STATEMENT: DELETE FROM sbtest3 WHERE id=$1 2025-09-01 03:52:40.354 GMT [252] LOG: duration: 2000.508 ms execute sbstmt-1494909615-1475726632: UPDATE sbtest3 SET c=$1 WHERE id=$2 2025-09-01 03:52:40.354 GMT [252] DETAIL: parameters: $1 = '46305822408-65061479249-32020893845-96941028398-55864827715-62349463850-77066472087-06553466791-72028832479-76508461527', $2 = '499' 2025-09-01 03:52:40.355 GMT [253] LOG: duration: 1999.400 ms execute sbstmt162655822652424470: DELETE FROM sbtest3 WHERE id=$1 2025-09-01 03:52:40.355 GMT [253] DETAIL: parameters: $1 = '504' 2025-09-01 03:52:43,935 INFO: no action. I am (orioledb-oihqvc-orioledb-0), the leader with the lock cluster expose check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster expose orioledb-oihqvc --auto-approve --force=true --type internet --enable false --components orioledb --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-expose-wzmv2 created successfully, you can view the progress: kbcli cluster describe-ops orioledb-oihqvc-expose-wzmv2 -n ns-eicpq check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-expose-wzmv2 ns-eicpq Expose orioledb-oihqvc Creating -/- Sep 01,2025 11:52 UTC+0800 check cluster status `kbcli cluster list orioledb-oihqvc --show-labels --namespace ns-eicpq ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS orioledb-oihqvc ns-eicpq orioledb orioledb-beta1 WipeOut Running Sep 01,2025 11:20 UTC+0800 app.kubernetes.io/instance=orioledb-oihqvc,clusterdefinition.kubeblocks.io/name=orioledb,clusterversion.kubeblocks.io/name=orioledb-beta1 check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances orioledb-oihqvc --namespace ns-eicpq ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME orioledb-oihqvc-orioledb-0 ns-eicpq orioledb-oihqvc orioledb Running primary 0 200m / 200m 644245094400m / 644245094400m data:15Gi aks-cicdamdpool-68727890-vmss000008/10.224.0.9 Sep 01,2025 11:44 UTC+0800 orioledb-oihqvc-orioledb-1 ns-eicpq orioledb-oihqvc orioledb Running secondary 0 200m / 200m 644245094400m / 644245094400m data:15Gi aks-cicdamdpool-68727890-vmss000006/10.224.0.10 Sep 01,2025 11:44 UTC+0800 check pod status done check cluster role check cluster role done primary: orioledb-oihqvc-orioledb-0;secondary: orioledb-oihqvc-orioledb-1 check cluster connect `echo '' | kbcli cluster connect orioledb-oihqvc --namespace ns-eicpq` check cluster connect done check ops status `kbcli cluster list-ops orioledb-oihqvc --status all --namespace ns-eicpq ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME orioledb-oihqvc-expose-wzmv2 ns-eicpq Expose orioledb-oihqvc orioledb Running 0/1 Sep 01,2025 11:52 UTC+0800 ops_status:orioledb-oihqvc-expose-wzmv2 ns-eicpq Expose orioledb-oihqvc orioledb Running 0/1 Sep 01,2025 11:52 UTC+0800 ops_status:orioledb-oihqvc-expose-wzmv2 ns-eicpq Expose orioledb-oihqvc orioledb Running 0/1 Sep 01,2025 11:52 UTC+0800 check ops status done ops_status:orioledb-oihqvc-expose-wzmv2 ns-eicpq Expose orioledb-oihqvc orioledb Succeed 1/1 Sep 01,2025 11:52 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests orioledb-oihqvc-expose-wzmv2 --namespace ns-eicpq ` opsrequest.apps.kubeblocks.io/orioledb-oihqvc-expose-wzmv2 patched `kbcli cluster delete-ops --name orioledb-oihqvc-expose-wzmv2 --force --auto-approve --namespace ns-eicpq ` OpsRequest orioledb-oihqvc-expose-wzmv2 deleted delete cluster orioledb-oihqvc `kbcli cluster delete orioledb-oihqvc --auto-approve --namespace ns-eicpq ` Cluster orioledb-oihqvc deleted pod_info:orioledb-oihqvc-orioledb-0 5/5 Running 0 8m21s orioledb-oihqvc-orioledb-1 5/5 Running 0 8m21s pod_info:orioledb-oihqvc-orioledb-0 4/5 Terminating 0 8m41s orioledb-oihqvc-orioledb-1 4/5 Terminating 0 8m41s No resources found in ns-eicpq namespace. delete cluster pod done No resources found in ns-eicpq namespace. check cluster resource non-exist OK: pvc No resources found in ns-eicpq namespace. delete cluster done No resources found in ns-eicpq namespace. No resources found in ns-eicpq namespace. No resources found in ns-eicpq namespace. Orioledb Test Suite All Done! delete cluster etcdo-oihqvc `kbcli cluster delete etcdo-oihqvc --auto-approve --namespace ns-eicpq ` Cluster etcdo-oihqvc deleted pod_info:etcdo-oihqvc-etcd-0 2/2 Running 0 36m etcdo-oihqvc-etcd-1 2/2 Running 0 35m etcdo-oihqvc-etcd-2 2/2 Running 0 34m No resources found in ns-eicpq namespace. delete cluster pod done No resources found in ns-eicpq namespace. check cluster resource non-exist OK: pvc No resources found in ns-eicpq namespace. delete cluster done No resources found in ns-eicpq namespace. No resources found in ns-eicpq namespace. No resources found in ns-eicpq namespace. [PASSED]|[Create]|[ClusterDefinition=etcd;]|[Description=Create a cluster with the specified cluster definition etcd] --------------------------------------Orioledb (Topology = Replicas 2) Test Result-------------------------------------- [PASSED]|[Create]|[ClusterDefinition=orioledb;ClusterVersion=orioledb-beta1;]|[Description=Create a cluster with the specified cluster definition orioledb and cluster version orioledb-beta1] [PASSED]|[Connect]|[ComponentName=orioledb]|[Description=Connect to the cluster] [PASSED]|[Expose]|[Enable=true;TYPE=internet;ComponentName=orioledb]|[Description=Expose Enable the internet service with orioledb component] [PASSED]|[VerticalScaling]|[ComponentName=orioledb]|[Description=VerticalScaling the cluster specify component orioledb] [PASSED]|[HorizontalScaling Out]|[ComponentName=orioledb]|[Description=HorizontalScaling Out the cluster specify component orioledb] [PASSED]|[HorizontalScaling In]|[ComponentName=orioledb]|[Description=HorizontalScaling In the cluster specify component orioledb] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [PASSED]|[VolumeExpansion]|[ComponentName=orioledb]|[Description=VolumeExpansion the cluster specify component orioledb] [PASSED]|[Stop]|[-]|[Description=Stop the cluster] [PASSED]|[Start]|[-]|[Description=Start the cluster] [PASSED]|[Bench]|[ComponentName=orioledb]|[Description=Bench the cluster service with orioledb component] [PASSED]|[Bench]|[HostType=LB;ComponentName=orioledb]|[Description=Bench the cluster LB service with orioledb component] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Expose]|[Disable=true;TYPE=internet;ComponentName=orioledb]|[Description=Expose Disable the internet service with orioledb component] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]