bash test/kbcli/test_kbcli_0.9.sh --type 34 --version 0.9.5 --generate-output true --chaos-mesh true --drain-node true --random-namespace true --region eastus --cloud-provider aks CURRENT_TEST_DIR:test/kbcli source commons files source engines files source kubeblocks files `kubectl get namespace | grep ns-shbjn ` `kubectl create namespace ns-shbjn` namespace/ns-shbjn created create namespace ns-shbjn done download kbcli `gh release list --repo apecloud/kbcli --limit 100 | (grep "0.9" || true)` `curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v0.9.5-beta.8` Your system is linux_amd64 Installing kbcli ... Downloading ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 32.1M 100 32.1M 0 0 64.6M 0 --:--:-- --:--:-- --:--:-- 64.6M kbcli installed successfully. Kubernetes: v1.32.6 KubeBlocks: 0.9.5 kbcli: 0.9.5-beta.8 WARNING: version difference between kbcli (0.9.5-beta.8) and kubeblocks (0.9.5) Make sure your docker service is running and begin your journey with kbcli: kbcli playground init For more information on how to get started, please visit: https://kubeblocks.io download kbcli v0.9.5-beta.8 done Kubernetes: v1.32.6 KubeBlocks: 0.9.5 kbcli: 0.9.5-beta.8 WARNING: version difference between kbcli (0.9.5-beta.8) and kubeblocks (0.9.5) Kubernetes Env: v1.32.6 POD_RESOURCES: aks kb-default-sc found aks default-vsc found found default storage class: default kubeblocks version is:0.9.5 skip upgrade kubeblocks Error: no repositories to show helm repo add chaos-mesh https://charts.chaos-mesh.org "chaos-mesh" has been added to your repositories add helm chart repo chaos-mesh success chaos mesh already installed check cluster definition set component name:tidb set component version set component version:tidb set service versions:7.1.5,7.5.2,8.4.0 set service versions sorted:7.1.5,7.5.2,8.4.0 no cluster version found unsupported component definition REPORT_COUNT 0:0 set replicas first:2 set replicas third:2 set minimum cmpv service version set minimum cmpv service version replicas:2 REPORT_COUNT:1 CLUSTER_TOPOLOGY: set cluster topology: cluster LIMIT_CPU:0.5 LIMIT_MEMORY:3 storage size: 20 No resources found in ns-shbjn namespace. termination_policy:WipeOut create 2 replica WipeOut tidb cluster check cluster definition check component definition set component definition by component version check cmpd by labels check cmpd by compDefs set component definition: tidb-7 by component version:tidb apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: name: tidb-khkrpa namespace: ns-shbjn spec: clusterDefinitionRef: tidb topology: cluster terminationPolicy: WipeOut componentSpecs: - name: tidb replicas: 2 serviceVersion: serviceAccountName: kb-tidb-khkrpa disableExporter: false resources: requests: cpu: 500m memory: 3Gi limits: cpu: 500m memory: 3Gi - name: tikv replicas: 1 serviceVersion: serviceAccountName: kb-tidb-khkrpa disableExporter: false resources: requests: cpu: 500m memory: 3Gi limits: cpu: 500m memory: 3Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi - name: tidb-pd replicas: 1 serviceVersion: serviceAccountName: kb-tidb-khkrpa disableExporter: false resources: requests: cpu: 500m memory: 3Gi limits: cpu: 500m memory: 3Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi `kubectl apply -f test_create_tidb-khkrpa.yaml` cluster.apps.kubeblocks.io/tidb-khkrpa created apply test_create_tidb-khkrpa.yaml Success `rm -rf test_create_tidb-khkrpa.yaml` check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Sep 01,2025 11:19 UTC+0800 clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 500m / 500m 3Gi / 3Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:30 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 500m / 500m 3Gi / 3Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:30 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 500m / 500m 3Gi / 3Gi data:20Gi aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:19 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 500m / 500m 3Gi / 3Gi data:20Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:25 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: check pod tidb-khkrpa-tidb-0 container_name tidb exist password F9Vkb4RtC9u2p2UN check pod tidb-khkrpa-tidb-0 container_name slowlog exist password F9Vkb4RtC9u2p2UN check pod tidb-khkrpa-tidb-0 container_name lorry exist password F9Vkb4RtC9u2p2UN No container logs contain secret password. describe cluster `kbcli cluster describe tidb-khkrpa --namespace ns-shbjn ` Name: tidb-khkrpa Created Time: Sep 01,2025 11:19 UTC+0800 NAMESPACE CLUSTER-DEFINITION VERSION STATUS TERMINATION-POLICY ns-shbjn tidb Running WipeOut Endpoints: COMPONENT MODE INTERNAL EXTERNAL tidb-pd ReadWrite tidb-khkrpa-tidb-pd.ns-shbjn.svc.cluster.local:2379 tidb-khkrpa-tidb-pd.ns-shbjn.svc.cluster.local:2380 tikv ReadWrite tidb-khkrpa-tikv.ns-shbjn.svc.cluster.local:20160 tidb-khkrpa-tikv.ns-shbjn.svc.cluster.local:20180 tidb ReadWrite tidb-khkrpa-tidb.ns-shbjn.svc.cluster.local:4000 tidb-khkrpa-tidb.ns-shbjn.svc.cluster.local:10080 Topology: COMPONENT INSTANCE ROLE STATUS AZ NODE CREATED-TIME tidb tidb-khkrpa-tidb-0 Running 0 aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:30 UTC+0800 tidb tidb-khkrpa-tidb-1 Running 0 aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:30 UTC+0800 tidb-pd tidb-khkrpa-tidb-pd-0 leader Running 0 aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:19 UTC+0800 tikv tidb-khkrpa-tikv-0 Running 0 aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:25 UTC+0800 Resources Allocation: COMPONENT DEDICATED CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS tidb-pd false 500m / 500m 3Gi / 3Gi data:20Gi default tikv false 500m / 500m 3Gi / 3Gi data:20Gi default tidb false 500m / 500m 3Gi / 3Gi Images: COMPONENT TYPE IMAGE tidb-pd docker.io/apecloud/pd:v8.4.0 tikv docker.io/apecloud/tikv:v8.4.0 tidb docker.io/apecloud/tidb:v8.4.0 Show cluster events: kbcli cluster list-events -n ns-shbjn tidb-khkrpa `kbcli cluster label tidb-khkrpa app.kubernetes.io/instance- --namespace ns-shbjn ` label "app.kubernetes.io/instance" not found. `kbcli cluster label tidb-khkrpa app.kubernetes.io/instance=tidb-khkrpa --namespace ns-shbjn ` `kbcli cluster label tidb-khkrpa --list --namespace ns-shbjn ` NAME NAMESPACE LABELS tidb-khkrpa ns-shbjn app.kubernetes.io/instance=tidb-khkrpa clusterdefinition.kubeblocks.io/name=tidb clusterversion.kubeblocks.io/name= label cluster app.kubernetes.io/instance=tidb-khkrpa Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=tidb-khkrpa --namespace ns-shbjn ` `kbcli cluster label tidb-khkrpa --list --namespace ns-shbjn ` NAME NAMESPACE LABELS tidb-khkrpa ns-shbjn app.kubernetes.io/instance=tidb-khkrpa case.name=kbcli.test1 clusterdefinition.kubeblocks.io/name=tidb clusterversion.kubeblocks.io/name= label cluster case.name=kbcli.test1 Success `kbcli cluster label tidb-khkrpa case.name=kbcli.test2 --overwrite --namespace ns-shbjn ` `kbcli cluster label tidb-khkrpa --list --namespace ns-shbjn ` NAME NAMESPACE LABELS tidb-khkrpa ns-shbjn app.kubernetes.io/instance=tidb-khkrpa case.name=kbcli.test2 clusterdefinition.kubeblocks.io/name=tidb clusterversion.kubeblocks.io/name= label cluster case.name=kbcli.test2 Success `kbcli cluster label tidb-khkrpa case.name- --namespace ns-shbjn ` `kbcli cluster label tidb-khkrpa --list --namespace ns-shbjn ` NAME NAMESPACE LABELS tidb-khkrpa ns-shbjn app.kubernetes.io/instance=tidb-khkrpa clusterdefinition.kubeblocks.io/name=tidb clusterversion.kubeblocks.io/name= delete cluster label case.name Success cluster connect create mysql-client pod tidb-khkrpa-mysql-client `kubectl run --namespace ns-shbjn tidb-khkrpa-mysql-client --restart='Never' --image docker.io/apecloud/mysql-client:latest --command -- sleep infinity` pod/tidb-khkrpa-mysql-client created check pod status pod_status:NAME READY STATUS RESTARTS AGE tidb-khkrpa-mysql-client 0/1 ContainerCreating 0 0s check pod tidb-khkrpa-mysql-client status done `kubectl wait --for=condition=Ready pod tidb-khkrpa-mysql-client --namespace ns-shbjn` pod/tidb-khkrpa-mysql-client condition met pod_status:NAME READY STATUS RESTARTS AGE tidb-khkrpa-mysql-client 1/1 Running 0 5s `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: `echo "echo 'select 1;' | mysql -P4000 -htidb-khkrpa-tidb.ns-shbjn.svc.cluster.local -uroot -p'F9Vkb4RtC9u2p2UN'" | kubectl exec -it tidb-khkrpa-mysql-client --namespace ns-shbjn -- sh ` Unable to use a TTY - input is not a terminal or the right kind of file 1 1 connect cluster Success insert batch data by db client Error from server (NotFound): pods "test-db-client-executionloop-tidb-khkrpa" not found DB_CLIENT_BATCH_DATA_COUNT: `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-tidb-khkrpa --namespace ns-shbjn ` Error from server (NotFound): pods "test-db-client-executionloop-tidb-khkrpa" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-executionloop-tidb-khkrpa" not found `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: apiVersion: v1 kind: Pod metadata: name: test-db-client-executionloop-tidb-khkrpa namespace: ns-shbjn spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "tidb-khkrpa-tidb.ns-shbjn.svc.cluster.local" - "--user" - "root" - "--password" - "F9Vkb4RtC9u2p2UN" - "--port" - "4000" - "--dbtype" - "tidb" - "--test" - "executionloop" - "--duration" - "60" - "--interval" - "1" restartPolicy: Never `kubectl apply -f test-db-client-executionloop-tidb-khkrpa.yaml` pod/test-db-client-executionloop-tidb-khkrpa created apply test-db-client-executionloop-tidb-khkrpa.yaml Success `rm -rf test-db-client-executionloop-tidb-khkrpa.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-tidb-khkrpa 1/1 Running 0 5s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-tidb-khkrpa 1/1 Running 0 10s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-tidb-khkrpa 1/1 Running 0 15s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-tidb-khkrpa 1/1 Running 0 21s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-tidb-khkrpa 1/1 Running 0 26s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-tidb-khkrpa 1/1 Running 0 32s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-tidb-khkrpa 1/1 Running 0 37s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-tidb-khkrpa 1/1 Running 0 43s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-tidb-khkrpa 1/1 Running 0 48s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-tidb-khkrpa 1/1 Running 0 54s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-tidb-khkrpa 1/1 Running 0 59s check pod test-db-client-executionloop-tidb-khkrpa status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-tidb-khkrpa 0/1 Completed 0 65s check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Running Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 500m / 500m 3Gi / 3Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:30 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 500m / 500m 3Gi / 3Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:30 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 500m / 500m 3Gi / 3Gi data:20Gi aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:19 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 500m / 500m 3Gi / 3Gi data:20Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:25 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 --host tidb-khkrpa-tidb.ns-shbjn.svc.cluster.local --user root --password F9Vkb4RtC9u2p2UN --port 4000 --dbtype tidb --test executionloop --duration 60 --interval 1 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider] Execution loop start: create databases executions_loop CREATE DATABASE IF NOT EXISTS executions_loop; drop table executions_loop_table DROP TABLE IF EXISTS executions_loop.executions_loop_table; create table executions_loop_table CREATE TABLE IF NOT EXISTS executions_loop.executions_loop_table (id INT PRIMARY KEY AUTO_INCREMENT, value VARCHAR(255), tinyint_col TINYINT, smallint_col SMALLINT, mediumint_col MEDIUMINT, int_col INT, bigint_col BIGINT, float_col FLOAT, double_col DOUBLE, decimal_col DECIMAL(10, 2), date_col DATE, time_col TIME, datetime_col DATETIME, timestamp_col TIMESTAMP, year_col YEAR, char_col CHAR(10), text_col TEXT, blob_col BLOB, binary_col BINARY(10), varbinary_col VARBINARY(255), enum_col ENUM('Option1', 'Option2', 'Option3'), set_col SET('Value1', 'Value2', 'Value3') ); Execution loop start:INSERT INTO executions_loop.executions_loop_table (value, tinyint_col, smallint_col, mediumint_col, int_col, bigint_col, float_col, double_col, decimal_col, date_col, time_col, datetime_col, timestamp_col, year_col, char_col, text_col, blob_col, binary_col, varbinary_col, enum_col, set_col) VALUES ('executions_loop_test_1', 46, 3489, 2005462, 2118909657, 6352302888829189572, 0.93922585, 0.2815044361793967, 4.640979391397493, '2025-09-01', '03:31:55', '2025-09-01 03:31:55.419', CURRENT_TIMESTAMP, 2006, 'HkhHifiT0I', 'rETFo4rUhKvNACD4pGguNh9hWvwd04CrArWi7xYiusVTEyxCE13gVzXQ9Mr64W1zbqeBStgyg8hwCESsmXgSzeYRYy4k38r5fr2HD5qqSlCs6SKpF7QKg7HXo5o17b9Nl6y1iS8jupSUwi2ssOykD2UesRd00kFCuvYHMLJORlRIKtzRas0PykRzPtjue2TBBBN7OLUcXtiI2bV14IY5IycFLthAOf2gzYg32ZlFMi33pNk72O4GkVSXic8GtED', UNHEX(REPLACE('1eb6fad0dd0d2a5fa8e2', ' ', '')), UNHEX(REPLACE('a8c12601d8c1c29ebef8', ' ', '')), UNHEX(REPLACE('a5ebdd9b82031c8e44bec77eff6bc543a1e96949296ba08c484536a43d967b71a498b89003b2c2d3be43f0e1040389d4ee06ca86ea1cd21b0f925bb0a7f054823a98c5761ef8b5501af3eeb253afa309477943107c5eac3a599d55b4625d1e609347bbf2f0bfd1e746b2dfe2ec94a97169f1cef127dbf500fb477891a4de00711e1215c4891e5a0b18e9bf39e0fd152a964e762e66a44082e8ae746cf96910370d7665cd6e3ef21dc6593fa4d488ba39136b7203664455adbd0b8f0ec17eef11d818343e191a96b26e89a3bc83a7757fa9f8851f8ca0fdc5362580d6af0cdf0fbaebd95d7406215b8fb87f48396dca98256d2793ecab5957b64fe48c4baa74', ' ', '')), 'Option2', 'Value3' ); [ 1s ] executions total: 98 successful: 98 failed: 0 disconnect: 0 [ 2s ] executions total: 257 successful: 257 failed: 0 disconnect: 0 [ 3s ] executions total: 417 successful: 417 failed: 0 disconnect: 0 [ 4s ] executions total: 586 successful: 586 failed: 0 disconnect: 0 [ 5s ] executions total: 758 successful: 758 failed: 0 disconnect: 0 [ 6s ] executions total: 929 successful: 929 failed: 0 disconnect: 0 [ 7s ] executions total: 1096 successful: 1096 failed: 0 disconnect: 0 [ 8s ] executions total: 1258 successful: 1258 failed: 0 disconnect: 0 [ 9s ] executions total: 1421 successful: 1421 failed: 0 disconnect: 0 [ 10s ] executions total: 1591 successful: 1591 failed: 0 disconnect: 0 [ 11s ] executions total: 1767 successful: 1767 failed: 0 disconnect: 0 [ 12s ] executions total: 1938 successful: 1938 failed: 0 disconnect: 0 [ 13s ] executions total: 2110 successful: 2110 failed: 0 disconnect: 0 [ 14s ] executions total: 2281 successful: 2281 failed: 0 disconnect: 0 [ 15s ] executions total: 2452 successful: 2452 failed: 0 disconnect: 0 [ 16s ] executions total: 2618 successful: 2618 failed: 0 disconnect: 0 [ 17s ] executions total: 2794 successful: 2794 failed: 0 disconnect: 0 [ 18s ] executions total: 2969 successful: 2969 failed: 0 disconnect: 0 [ 19s ] executions total: 3141 successful: 3141 failed: 0 disconnect: 0 [ 20s ] executions total: 3321 successful: 3321 failed: 0 disconnect: 0 [ 21s ] executions total: 3495 successful: 3495 failed: 0 disconnect: 0 [ 22s ] executions total: 3670 successful: 3670 failed: 0 disconnect: 0 [ 23s ] executions total: 3851 successful: 3851 failed: 0 disconnect: 0 [ 24s ] executions total: 4025 successful: 4025 failed: 0 disconnect: 0 [ 25s ] executions total: 4190 successful: 4190 failed: 0 disconnect: 0 [ 26s ] executions total: 4369 successful: 4369 failed: 0 disconnect: 0 [ 27s ] executions total: 4551 successful: 4551 failed: 0 disconnect: 0 [ 28s ] executions total: 4729 successful: 4729 failed: 0 disconnect: 0 [ 29s ] executions total: 4902 successful: 4902 failed: 0 disconnect: 0 [ 30s ] executions total: 5078 successful: 5078 failed: 0 disconnect: 0 [ 31s ] executions total: 5234 successful: 5234 failed: 0 disconnect: 0 [ 32s ] executions total: 5407 successful: 5407 failed: 0 disconnect: 0 [ 33s ] executions total: 5579 successful: 5579 failed: 0 disconnect: 0 [ 34s ] executions total: 5757 successful: 5757 failed: 0 disconnect: 0 [ 35s ] executions total: 5934 successful: 5934 failed: 0 disconnect: 0 [ 36s ] executions total: 6106 successful: 6106 failed: 0 disconnect: 0 [ 37s ] executions total: 6280 successful: 6280 failed: 0 disconnect: 0 [ 38s ] executions total: 6452 successful: 6452 failed: 0 disconnect: 0 [ 39s ] executions total: 6623 successful: 6623 failed: 0 disconnect: 0 [ 40s ] executions total: 6781 successful: 6781 failed: 0 disconnect: 0 [ 41s ] executions total: 6955 successful: 6955 failed: 0 disconnect: 0 [ 42s ] executions total: 7131 successful: 7131 failed: 0 disconnect: 0 [ 43s ] executions total: 7308 successful: 7308 failed: 0 disconnect: 0 [ 44s ] executions total: 7479 successful: 7479 failed: 0 disconnect: 0 [ 45s ] executions total: 7650 successful: 7650 failed: 0 disconnect: 0 [ 46s ] executions total: 7808 successful: 7808 failed: 0 disconnect: 0 [ 47s ] executions total: 7983 successful: 7983 failed: 0 disconnect: 0 [ 48s ] executions total: 8160 successful: 8160 failed: 0 disconnect: 0 [ 49s ] executions total: 8335 successful: 8335 failed: 0 disconnect: 0 [ 50s ] executions total: 8510 successful: 8510 failed: 0 disconnect: 0 [ 51s ] executions total: 8685 successful: 8685 failed: 0 disconnect: 0 [ 52s ] executions total: 8857 successful: 8857 failed: 0 disconnect: 0 [ 53s ] executions total: 9036 successful: 9036 failed: 0 disconnect: 0 [ 54s ] executions total: 9212 successful: 9212 failed: 0 disconnect: 0 [ 55s ] executions total: 9390 successful: 9390 failed: 0 disconnect: 0 [ 56s ] executions total: 9568 successful: 9568 failed: 0 disconnect: 0 [ 57s ] executions total: 9745 successful: 9745 failed: 0 disconnect: 0 [ 58s ] executions total: 9916 successful: 9916 failed: 0 disconnect: 0 [ 59s ] executions total: 10092 successful: 10092 failed: 0 disconnect: 0 [ 60s ] executions total: 10227 successful: 10227 failed: 0 disconnect: 0 Test Result: Total Executions: 10227 Successful Executions: 10227 Failed Executions: 0 Disconnection Counts: 0 Connection Information: Database Type: tidb Host: tidb-khkrpa-tidb.ns-shbjn.svc.cluster.local Port: 4000 Database: Table: User: root Org: Access Mode: mysql Test Type: executionloop Query: Duration: 60 seconds Interval: 1 seconds DB_CLIENT_BATCH_DATA_COUNT: 10227 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-tidb-khkrpa --namespace ns-shbjn ` pod/test-db-client-executionloop-tidb-khkrpa patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-executionloop-tidb-khkrpa" force deleted check component tikv exists `kubectl get components -l app.kubernetes.io/instance=tidb-khkrpa,apps.kubeblocks.io/component-name=tikv --namespace ns-shbjn | (grep "tikv" || true )` cluster hscale check cluster status before ops check cluster status done cluster_status:Running No resources found in tidb-khkrpa namespace. `kbcli cluster hscale tidb-khkrpa --auto-approve --force=true --components tikv --replicas 2 --namespace ns-shbjn ` OpsRequest tidb-khkrpa-horizontalscaling-kmfxg created successfully, you can view the progress: kbcli cluster describe-ops tidb-khkrpa-horizontalscaling-kmfxg -n ns-shbjn check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-horizontalscaling-kmfxg ns-shbjn HorizontalScaling tidb-khkrpa tikv Running 0/1 Sep 01,2025 11:33 UTC+0800 check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Updating Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 500m / 500m 3Gi / 3Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:30 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 500m / 500m 3Gi / 3Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:30 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 500m / 500m 3Gi / 3Gi data:20Gi aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:19 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 500m / 500m 3Gi / 3Gi data:20Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:25 UTC+0800 tidb-khkrpa-tikv-1 ns-shbjn tidb-khkrpa tikv Running 0 500m / 500m 3Gi / 3Gi data:20Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:33 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 No resources found in tidb-khkrpa namespace. check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-horizontalscaling-kmfxg ns-shbjn HorizontalScaling tidb-khkrpa tikv Succeed 1/1 Sep 01,2025 11:33 UTC+0800 check ops status done ops_status:tidb-khkrpa-horizontalscaling-kmfxg ns-shbjn HorizontalScaling tidb-khkrpa tikv Succeed 1/1 Sep 01,2025 11:33 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests tidb-khkrpa-horizontalscaling-kmfxg --namespace ns-shbjn ` opsrequest.apps.kubeblocks.io/tidb-khkrpa-horizontalscaling-kmfxg patched `kbcli cluster delete-ops --name tidb-khkrpa-horizontalscaling-kmfxg --force --auto-approve --namespace ns-shbjn ` OpsRequest tidb-khkrpa-horizontalscaling-kmfxg deleted `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P4000 -htidb-khkrpa-tidb.ns-shbjn.svc.cluster.local -uroot -p'F9Vkb4RtC9u2p2UN' " | kubectl exec -it tidb-khkrpa-mysql-client --namespace ns-shbjn -- sh ` check db_client batch data Success check component tidb-pd exists `kubectl get components -l app.kubernetes.io/instance=tidb-khkrpa,apps.kubeblocks.io/component-name=tidb-pd --namespace ns-shbjn | (grep "tidb-pd" || true )` check component tikv exists `kubectl get components -l app.kubernetes.io/instance=tidb-khkrpa,apps.kubeblocks.io/component-name=tikv --namespace ns-shbjn | (grep "tikv" || true )` `kubectl get pvc -l app.kubernetes.io/instance=tidb-khkrpa,apps.kubeblocks.io/component-name=tidb-pd,tikv,apps.kubeblocks.io/vct-name=data --namespace ns-shbjn ` No resources found in ns-shbjn namespace. tidb-khkrpa tidb-pd,tikv data pvc is empty cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in tidb-khkrpa namespace. `kbcli cluster volume-expand tidb-khkrpa --auto-approve --force=true --components tidb-pd,tikv --volume-claim-templates data --storage 21Gi --namespace ns-shbjn ` OpsRequest tidb-khkrpa-volumeexpansion-zdbgr created successfully, you can view the progress: kbcli cluster describe-ops tidb-khkrpa-volumeexpansion-zdbgr -n ns-shbjn check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-volumeexpansion-zdbgr ns-shbjn VolumeExpansion tidb-khkrpa tidb-pd,tikv Running 0/3 Sep 01,2025 11:33 UTC+0800 check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Updating Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 500m / 500m 3Gi / 3Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:30 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 500m / 500m 3Gi / 3Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:30 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 500m / 500m 3Gi / 3Gi data:21Gi aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:19 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 500m / 500m 3Gi / 3Gi data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:25 UTC+0800 tidb-khkrpa-tikv-1 ns-shbjn tidb-khkrpa tikv Running 0 500m / 500m 3Gi / 3Gi data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:33 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 No resources found in tidb-khkrpa namespace. check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-volumeexpansion-zdbgr ns-shbjn VolumeExpansion tidb-khkrpa tidb-pd,tikv Succeed 3/3 Sep 01,2025 11:33 UTC+0800 check ops status done ops_status:tidb-khkrpa-volumeexpansion-zdbgr ns-shbjn VolumeExpansion tidb-khkrpa tidb-pd,tikv Succeed 3/3 Sep 01,2025 11:33 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests tidb-khkrpa-volumeexpansion-zdbgr --namespace ns-shbjn ` opsrequest.apps.kubeblocks.io/tidb-khkrpa-volumeexpansion-zdbgr patched `kbcli cluster delete-ops --name tidb-khkrpa-volumeexpansion-zdbgr --force --auto-approve --namespace ns-shbjn ` OpsRequest tidb-khkrpa-volumeexpansion-zdbgr deleted `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P4000 -htidb-khkrpa-tidb.ns-shbjn.svc.cluster.local -uroot -p'F9Vkb4RtC9u2p2UN' " | kubectl exec -it tidb-khkrpa-mysql-client --namespace ns-shbjn -- sh ` check db_client batch data Success cluster stop check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster stop tidb-khkrpa --auto-approve --force=true --namespace ns-shbjn ` OpsRequest tidb-khkrpa-stop-8dlhg created successfully, you can view the progress: kbcli cluster describe-ops tidb-khkrpa-stop-8dlhg -n ns-shbjn check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-stop-8dlhg ns-shbjn Stop tidb-khkrpa tidb,tidb-pd,tikv Running 0/5 Sep 01,2025 11:40 UTC+0800 check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Stopping Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping check cluster status done cluster_status:Stopped check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME check pod status done check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-stop-8dlhg ns-shbjn Stop tidb-khkrpa tidb,tidb-pd,tikv Succeed 5/5 Sep 01,2025 11:40 UTC+0800 check ops status done ops_status:tidb-khkrpa-stop-8dlhg ns-shbjn Stop tidb-khkrpa tidb,tidb-pd,tikv Succeed 5/5 Sep 01,2025 11:40 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests tidb-khkrpa-stop-8dlhg --namespace ns-shbjn ` opsrequest.apps.kubeblocks.io/tidb-khkrpa-stop-8dlhg patched `kbcli cluster delete-ops --name tidb-khkrpa-stop-8dlhg --force --auto-approve --namespace ns-shbjn ` OpsRequest tidb-khkrpa-stop-8dlhg deleted cluster start check cluster status before ops check cluster status done cluster_status:Stopped `kbcli cluster start tidb-khkrpa --force=true --namespace ns-shbjn ` OpsRequest tidb-khkrpa-start-r8hwk created successfully, you can view the progress: kbcli cluster describe-ops tidb-khkrpa-start-r8hwk -n ns-shbjn check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-start-r8hwk ns-shbjn Start tidb-khkrpa tidb,tidb-pd,tikv Running 0/5 Sep 01,2025 11:41 UTC+0800 check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Updating Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 500m / 500m 3Gi / 3Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:41 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 500m / 500m 3Gi / 3Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:41 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 500m / 500m 3Gi / 3Gi data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:41 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 500m / 500m 3Gi / 3Gi data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:41 UTC+0800 tidb-khkrpa-tikv-1 ns-shbjn tidb-khkrpa tikv Running 0 500m / 500m 3Gi / 3Gi data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:41 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-start-r8hwk ns-shbjn Start tidb-khkrpa tidb,tidb-pd,tikv Succeed 5/5 Sep 01,2025 11:41 UTC+0800 check ops status done ops_status:tidb-khkrpa-start-r8hwk ns-shbjn Start tidb-khkrpa tidb,tidb-pd,tikv Succeed 5/5 Sep 01,2025 11:41 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests tidb-khkrpa-start-r8hwk --namespace ns-shbjn ` opsrequest.apps.kubeblocks.io/tidb-khkrpa-start-r8hwk patched `kbcli cluster delete-ops --name tidb-khkrpa-start-r8hwk --force --auto-approve --namespace ns-shbjn ` OpsRequest tidb-khkrpa-start-r8hwk deleted `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P4000 -htidb-khkrpa-tidb.ns-shbjn.svc.cluster.local -uroot -p'F9Vkb4RtC9u2p2UN' " | kubectl exec -it tidb-khkrpa-mysql-client --namespace ns-shbjn -- sh ` check db_client batch data retry times: 1 check db_client batch data retry times: 2 check db_client batch data retry times: 3 check db_client batch data Success cluster vscale check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale tidb-khkrpa --auto-approve --force=true --components tidb --cpu 600m --memory 3.1Gi --namespace ns-shbjn ` OpsRequest tidb-khkrpa-verticalscaling-wjfcq created successfully, you can view the progress: kbcli cluster describe-ops tidb-khkrpa-verticalscaling-wjfcq -n ns-shbjn check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-verticalscaling-wjfcq ns-shbjn VerticalScaling tidb-khkrpa tidb Running 0/2 Sep 01,2025 11:43 UTC+0800 check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Updating Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:44 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:43 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 500m / 500m 3Gi / 3Gi data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:41 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 500m / 500m 3Gi / 3Gi data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:41 UTC+0800 tidb-khkrpa-tikv-1 ns-shbjn tidb-khkrpa tikv Running 0 500m / 500m 3Gi / 3Gi data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:41 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-verticalscaling-wjfcq ns-shbjn VerticalScaling tidb-khkrpa tidb Succeed 2/2 Sep 01,2025 11:43 UTC+0800 check ops status done ops_status:tidb-khkrpa-verticalscaling-wjfcq ns-shbjn VerticalScaling tidb-khkrpa tidb Succeed 2/2 Sep 01,2025 11:43 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests tidb-khkrpa-verticalscaling-wjfcq --namespace ns-shbjn ` opsrequest.apps.kubeblocks.io/tidb-khkrpa-verticalscaling-wjfcq patched `kbcli cluster delete-ops --name tidb-khkrpa-verticalscaling-wjfcq --force --auto-approve --namespace ns-shbjn ` OpsRequest tidb-khkrpa-verticalscaling-wjfcq deleted `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P4000 -htidb-khkrpa-tidb.ns-shbjn.svc.cluster.local -uroot -p'F9Vkb4RtC9u2p2UN' " | kubectl exec -it tidb-khkrpa-mysql-client --namespace ns-shbjn -- sh ` check db_client batch data Success check component tidb-pd exists `kubectl get components -l app.kubernetes.io/instance=tidb-khkrpa,apps.kubeblocks.io/component-name=tidb-pd --namespace ns-shbjn | (grep "tidb-pd" || true )` check component tikv exists `kubectl get components -l app.kubernetes.io/instance=tidb-khkrpa,apps.kubeblocks.io/component-name=tikv --namespace ns-shbjn | (grep "tikv" || true )` cluster vscale check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale tidb-khkrpa --auto-approve --force=true --components tidb-pd,tikv --cpu 600m --memory 3.1Gi --namespace ns-shbjn ` OpsRequest tidb-khkrpa-verticalscaling-s9rbg created successfully, you can view the progress: kbcli cluster describe-ops tidb-khkrpa-verticalscaling-s9rbg -n ns-shbjn check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-verticalscaling-s9rbg ns-shbjn VerticalScaling tidb-khkrpa tidb-pd,tikv Running 0/3 Sep 01,2025 11:44 UTC+0800 check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Updating Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:44 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:43 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:44 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:50 UTC+0800 tidb-khkrpa-tikv-1 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:44 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-verticalscaling-s9rbg ns-shbjn VerticalScaling tidb-khkrpa tidb-pd,tikv Succeed 3/3 Sep 01,2025 11:44 UTC+0800 check ops status done ops_status:tidb-khkrpa-verticalscaling-s9rbg ns-shbjn VerticalScaling tidb-khkrpa tidb-pd,tikv Succeed 3/3 Sep 01,2025 11:44 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests tidb-khkrpa-verticalscaling-s9rbg --namespace ns-shbjn ` opsrequest.apps.kubeblocks.io/tidb-khkrpa-verticalscaling-s9rbg patched `kbcli cluster delete-ops --name tidb-khkrpa-verticalscaling-s9rbg --force --auto-approve --namespace ns-shbjn ` OpsRequest tidb-khkrpa-verticalscaling-s9rbg deleted `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P4000 -htidb-khkrpa-tidb.ns-shbjn.svc.cluster.local -uroot -p'F9Vkb4RtC9u2p2UN' " | kubectl exec -it tidb-khkrpa-mysql-client --namespace ns-shbjn -- sh ` check db_client batch data Success cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart tidb-khkrpa --auto-approve --force=true --namespace ns-shbjn ` OpsRequest tidb-khkrpa-restart-64gfx created successfully, you can view the progress: kbcli cluster describe-ops tidb-khkrpa-restart-64gfx -n ns-shbjn check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-restart-64gfx ns-shbjn Restart tidb-khkrpa tidb-pd,tikv,tidb Running 0/5 Sep 01,2025 11:51 UTC+0800 check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Updating Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:52 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:52 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:51 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:52 UTC+0800 tidb-khkrpa-tikv-1 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:51 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-restart-64gfx ns-shbjn Restart tidb-khkrpa tidb-pd,tikv,tidb Succeed 5/5 Sep 01,2025 11:51 UTC+0800 check ops status done ops_status:tidb-khkrpa-restart-64gfx ns-shbjn Restart tidb-khkrpa tidb-pd,tikv,tidb Succeed 5/5 Sep 01,2025 11:51 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests tidb-khkrpa-restart-64gfx --namespace ns-shbjn ` opsrequest.apps.kubeblocks.io/tidb-khkrpa-restart-64gfx patched `kbcli cluster delete-ops --name tidb-khkrpa-restart-64gfx --force --auto-approve --namespace ns-shbjn ` OpsRequest tidb-khkrpa-restart-64gfx deleted `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P4000 -htidb-khkrpa-tidb.ns-shbjn.svc.cluster.local -uroot -p'F9Vkb4RtC9u2p2UN' " | kubectl exec -it tidb-khkrpa-mysql-client --namespace ns-shbjn -- sh ` check db_client batch data Success check component tidb exists `kubectl get components -l app.kubernetes.io/instance=tidb-khkrpa,apps.kubeblocks.io/component-name=tidb --namespace ns-shbjn | (grep "tidb" || true )` check component tikv exists `kubectl get components -l app.kubernetes.io/instance=tidb-khkrpa,apps.kubeblocks.io/component-name=tikv --namespace ns-shbjn | (grep "tikv" || true )` cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart tidb-khkrpa --auto-approve --force=true --components tidb,tikv --namespace ns-shbjn ` OpsRequest tidb-khkrpa-restart-c7jwf created successfully, you can view the progress: kbcli cluster describe-ops tidb-khkrpa-restart-c7jwf -n ns-shbjn check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-restart-c7jwf ns-shbjn Restart tidb-khkrpa tidb,tikv Running 0/4 Sep 01,2025 11:53 UTC+0800 check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Updating Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:54 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:53 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:51 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:53 UTC+0800 tidb-khkrpa-tikv-1 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:53 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-restart-c7jwf ns-shbjn Restart tidb-khkrpa tidb,tikv Succeed 4/4 Sep 01,2025 11:53 UTC+0800 check ops status done ops_status:tidb-khkrpa-restart-c7jwf ns-shbjn Restart tidb-khkrpa tidb,tikv Succeed 4/4 Sep 01,2025 11:53 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests tidb-khkrpa-restart-c7jwf --namespace ns-shbjn ` opsrequest.apps.kubeblocks.io/tidb-khkrpa-restart-c7jwf patched `kbcli cluster delete-ops --name tidb-khkrpa-restart-c7jwf --force --auto-approve --namespace ns-shbjn ` OpsRequest tidb-khkrpa-restart-c7jwf deleted `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P4000 -htidb-khkrpa-tidb.ns-shbjn.svc.cluster.local -uroot -p'F9Vkb4RtC9u2p2UN' " | kubectl exec -it tidb-khkrpa-mysql-client --namespace ns-shbjn -- sh ` check db_client batch data Success cluster hscale check cluster status before ops check cluster status done cluster_status:Running No resources found in tidb-khkrpa namespace. `kbcli cluster hscale tidb-khkrpa --auto-approve --force=true --components tidb --replicas 3 --namespace ns-shbjn ` OpsRequest tidb-khkrpa-horizontalscaling-6tnzf created successfully, you can view the progress: kbcli cluster describe-ops tidb-khkrpa-horizontalscaling-6tnzf -n ns-shbjn check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-horizontalscaling-6tnzf ns-shbjn HorizontalScaling tidb-khkrpa tidb Running 0/1 Sep 01,2025 11:54 UTC+0800 check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Updating Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:54 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:53 UTC+0800 tidb-khkrpa-tidb-2 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:54 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:51 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:53 UTC+0800 tidb-khkrpa-tikv-1 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:53 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 No resources found in tidb-khkrpa namespace. check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-horizontalscaling-6tnzf ns-shbjn HorizontalScaling tidb-khkrpa tidb Succeed 1/1 Sep 01,2025 11:54 UTC+0800 check ops status done ops_status:tidb-khkrpa-horizontalscaling-6tnzf ns-shbjn HorizontalScaling tidb-khkrpa tidb Succeed 1/1 Sep 01,2025 11:54 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests tidb-khkrpa-horizontalscaling-6tnzf --namespace ns-shbjn ` opsrequest.apps.kubeblocks.io/tidb-khkrpa-horizontalscaling-6tnzf patched `kbcli cluster delete-ops --name tidb-khkrpa-horizontalscaling-6tnzf --force --auto-approve --namespace ns-shbjn ` OpsRequest tidb-khkrpa-horizontalscaling-6tnzf deleted `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P4000 -htidb-khkrpa-tidb.ns-shbjn.svc.cluster.local -uroot -p'F9Vkb4RtC9u2p2UN' " | kubectl exec -it tidb-khkrpa-mysql-client --namespace ns-shbjn -- sh ` check db_client batch data Success cluster hscale check cluster status before ops check cluster status done cluster_status:Running No resources found in tidb-khkrpa namespace. `kbcli cluster hscale tidb-khkrpa --auto-approve --force=true --components tidb --replicas 2 --namespace ns-shbjn ` OpsRequest tidb-khkrpa-horizontalscaling-77npd created successfully, you can view the progress: kbcli cluster describe-ops tidb-khkrpa-horizontalscaling-77npd -n ns-shbjn check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-horizontalscaling-77npd ns-shbjn HorizontalScaling tidb-khkrpa tidb Running 0/1 Sep 01,2025 11:55 UTC+0800 check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Updating Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:54 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:53 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:51 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:53 UTC+0800 tidb-khkrpa-tikv-1 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:53 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 No resources found in tidb-khkrpa namespace. check ops status `kbcli cluster list-ops tidb-khkrpa --status all --namespace ns-shbjn ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME tidb-khkrpa-horizontalscaling-77npd ns-shbjn HorizontalScaling tidb-khkrpa tidb Succeed 1/1 Sep 01,2025 11:55 UTC+0800 check ops status done ops_status:tidb-khkrpa-horizontalscaling-77npd ns-shbjn HorizontalScaling tidb-khkrpa tidb Succeed 1/1 Sep 01,2025 11:55 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests tidb-khkrpa-horizontalscaling-77npd --namespace ns-shbjn ` opsrequest.apps.kubeblocks.io/tidb-khkrpa-horizontalscaling-77npd patched `kbcli cluster delete-ops --name tidb-khkrpa-horizontalscaling-77npd --force --auto-approve --namespace ns-shbjn ` OpsRequest tidb-khkrpa-horizontalscaling-77npd deleted `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P4000 -htidb-khkrpa-tidb.ns-shbjn.svc.cluster.local -uroot -p'F9Vkb4RtC9u2p2UN' " | kubectl exec -it tidb-khkrpa-mysql-client --namespace ns-shbjn -- sh ` check db_client batch data Success test failover connectionstress check cluster status before cluster-failover-connectionstress check cluster status done cluster_status:Running check node drain check node drain success Error from server (NotFound): pods "test-db-client-connectionstress-tidb-khkrpa" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstress-tidb-khkrpa --namespace ns-shbjn ` Error from server (NotFound): pods "test-db-client-connectionstress-tidb-khkrpa" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-connectionstress-tidb-khkrpa" not found `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: apiVersion: v1 kind: Pod metadata: name: test-db-client-connectionstress-tidb-khkrpa namespace: ns-shbjn spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "tidb-khkrpa-tidb.ns-shbjn.svc.cluster.local" - "--user" - "root" - "--password" - "F9Vkb4RtC9u2p2UN" - "--port" - "4000" - "--database" - "" - "--dbtype" - "tidb" - "--test" - "connectionstress" - "--connections" - "200" - "--duration" - "60" restartPolicy: Never `kubectl apply -f test-db-client-connectionstress-tidb-khkrpa.yaml` pod/test-db-client-connectionstress-tidb-khkrpa created apply test-db-client-connectionstress-tidb-khkrpa.yaml Success `rm -rf test-db-client-connectionstress-tidb-khkrpa.yaml` check pod status check pod test-db-client-connectionstress-tidb-khkrpa status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-tidb-khkrpa 0/1 Completed 0 5s check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Running Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:54 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:53 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:51 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:53 UTC+0800 tidb-khkrpa-tikv-1 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:53 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 --host tidb-khkrpa-tidb.ns-shbjn.svc.cluster.local --user root --password F9Vkb4RtC9u2p2UN --port 4000 --database --dbtype tidb --test connectionstress --connections 200 --duration 60 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider] Test Result: null Connection Information: Database Type: tidb Host: tidb-khkrpa-tidb.ns-shbjn.svc.cluster.local Port: 4000 Database: Table: User: root Org: Access Mode: mysql Test Type: connectionstress Connection Count: 200 Duration: 60 seconds `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstress-tidb-khkrpa --namespace ns-shbjn ` pod/test-db-client-connectionstress-tidb-khkrpa patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-connectionstress-tidb-khkrpa" force deleted check failover pod name failover pod name:tidb-khkrpa-tidb-0 failover connectionstress Success `kubectl get secrets -l app.kubernetes.io/instance=tidb-khkrpa` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.username***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.password***"` `kubectl get secrets tidb-khkrpa-tidb-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:F9Vkb4RtC9u2p2UN;DB_PORT:4000;DB_DATABASE: check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P4000 -htidb-khkrpa-tidb.ns-shbjn.svc.cluster.local -uroot -p'F9Vkb4RtC9u2p2UN' " | kubectl exec -it tidb-khkrpa-mysql-client --namespace ns-shbjn -- sh ` check db_client batch data Success cluster update terminationPolicy WipeOut `kbcli cluster update tidb-khkrpa --termination-policy=WipeOut --namespace ns-shbjn ` cluster.apps.kubeblocks.io/tidb-khkrpa updated (no change) check cluster status `kbcli cluster list tidb-khkrpa --show-labels --namespace ns-shbjn ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS tidb-khkrpa ns-shbjn tidb WipeOut Running Sep 01,2025 11:19 UTC+0800 app.kubernetes.io/instance=tidb-khkrpa,clusterdefinition.kubeblocks.io/name=tidb,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances tidb-khkrpa --namespace ns-shbjn ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME tidb-khkrpa-tidb-0 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:54 UTC+0800 tidb-khkrpa-tidb-1 ns-shbjn tidb-khkrpa tidb Running 0 600m / 600m 3328599654400m / 3328599654400m aks-cicdamdpool-15164480-vmss000000/10.224.0.5 Sep 01,2025 11:53 UTC+0800 tidb-khkrpa-tidb-pd-0 ns-shbjn tidb-khkrpa tidb-pd Running leader 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:51 UTC+0800 tidb-khkrpa-tikv-0 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:53 UTC+0800 tidb-khkrpa-tikv-1 ns-shbjn tidb-khkrpa tikv Running 0 600m / 600m 3328599654400m / 3328599654400m data:21Gi aks-cicdamdpool-15164480-vmss000005/10.224.0.6 Sep 01,2025 11:53 UTC+0800 check pod status done connect unsupported engine type: tidb tidb-7 cluster list-logs `kbcli cluster list-logs tidb-khkrpa --namespace ns-shbjn ` No log files found. You can enable the log feature with the kbcli command below. kbcli cluster update tidb-khkrpa --enable-all-logs=true --namespace ns-shbjn Error from server (NotFound): pods "tidb-khkrpa-tidb-0" not found cluster logs `kbcli cluster logs tidb-khkrpa --tail 30 --namespace ns-shbjn ` Defaulted container "pd" out of: pd, lorry, config-manager, init-lorry (init), init-pd-ctl (init) [2025/09/01 03:54:05.548 +00:00] [INFO] [region.go:765] ["leader changed"] [region-id=94] [from=142] [to=1] [2025/09/01 03:54:05.553 +00:00] [INFO] [region.go:765] ["leader changed"] [region-id=122] [from=142] [to=1] [2025/09/01 03:54:06.635 +00:00] [WARN] [proxy.go:193] ["fail to recv activity from remote, stay inactive and wait to next checking round"] [remote=tidb-khkrpa-tidb-1.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:4000] [interval=2s] [error="dial tcp 10.244.4.32:4000: connect: connection refused"] [2025/09/01 03:54:06.635 +00:00] [WARN] [proxy.go:193] ["fail to recv activity from remote, stay inactive and wait to next checking round"] [remote=tidb-khkrpa-tidb-1.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:10080] [interval=2s] [error="dial tcp 10.244.4.32:10080: connect: connection refused"] [2025/09/01 03:54:07.552 +00:00] [INFO] [region.go:765] ["leader changed"] [region-id=80] [from=142] [to=1] [2025/09/01 03:54:08.479 +00:00] [INFO] [region.go:765] ["leader changed"] [region-id=128] [from=1] [to=142] [2025/09/01 03:54:08.482 +00:00] [INFO] [region.go:765] ["leader changed"] [region-id=134] [from=1] [to=142] [2025/09/01 03:54:08.484 +00:00] [INFO] [region.go:765] ["leader changed"] [region-id=60] [from=1] [to=142] [2025/09/01 03:54:08.632 +00:00] [WARN] [proxy.go:193] ["fail to recv activity from remote, stay inactive and wait to next checking round"] [remote=tidb-khkrpa-tidb-1.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:4000] [interval=2s] [error="dial tcp 10.244.4.32:4000: connect: connection refused"] [2025/09/01 03:54:08.632 +00:00] [WARN] [proxy.go:193] ["fail to recv activity from remote, stay inactive and wait to next checking round"] [remote=tidb-khkrpa-tidb-1.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:10080] [interval=2s] [error="dial tcp 10.244.4.32:10080: connect: connection refused"] [2025/09/01 03:54:09.554 +00:00] [INFO] [region.go:765] ["leader changed"] [region-id=40] [from=142] [to=1] [2025/09/01 03:54:09.674 +00:00] [INFO] [grpc_service.go:2267] ["update service GC safe point"] [service-id=gc_worker] [expire-at=-9223372035098076960] [safepoint=460507905955725312] [2025/09/01 03:54:10.631 +00:00] [WARN] [proxy.go:193] ["fail to recv activity from remote, stay inactive and wait to next checking round"] [remote=tidb-khkrpa-tidb-1.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:4000] [interval=2s] [error="dial tcp 10.244.4.32:4000: connect: connection refused"] [2025/09/01 03:54:10.631 +00:00] [WARN] [proxy.go:193] ["fail to recv activity from remote, stay inactive and wait to next checking round"] [remote=tidb-khkrpa-tidb-1.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:10080] [interval=2s] [error="dial tcp 10.244.4.32:10080: connect: connection refused"] [2025/09/01 03:54:11.485 +00:00] [INFO] [region.go:765] ["leader changed"] [region-id=20] [from=1] [to=142] [2025/09/01 03:54:11.487 +00:00] [INFO] [region.go:765] ["leader changed"] [region-id=8] [from=1] [to=142] [2025/09/01 03:54:12.641 +00:00] [WARN] [proxy.go:193] ["fail to recv activity from remote, stay inactive and wait to next checking round"] [remote=tidb-khkrpa-tidb-1.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:10080] [interval=2s] [error="dial tcp 10.244.4.32:10080: connect: connection refused"] [2025/09/01 03:54:12.641 +00:00] [WARN] [proxy.go:193] ["fail to recv activity from remote, stay inactive and wait to next checking round"] [remote=tidb-khkrpa-tidb-1.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:4000] [interval=2s] [error="dial tcp 10.244.4.32:4000: connect: connection refused"] [2025/09/01 03:54:20.627 +00:00] [WARN] [tidb.go:65] ["Alive of TiDB has expired, maybe local time in different hosts are not synchronized"] [key=/topology/tidb/tidb-khkrpa-tidb-0.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:4000/ttl] [value=1756698813901421026] [2025/09/01 03:54:25.629 +00:00] [WARN] [tidb.go:65] ["Alive of TiDB has expired, maybe local time in different hosts are not synchronized"] [key=/topology/tidb/tidb-khkrpa-tidb-0.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:4000/ttl] [value=1756698813901421026] [2025/09/01 03:54:29.273 +00:00] [INFO] [grpc_service.go:99] ["watch request"] [key=resource_group/controller] [range-end=resource_group/controlles] [start-revision=1310] [2025/09/01 03:54:29.273 +00:00] [INFO] [grpc_service.go:99] ["watch request"] [key=resource_group/settings] [range-end=resource_group/settingt] [start-revision=1310] [2025/09/01 03:54:30.614 +00:00] [WARN] [proxy.go:143] ["remote become inactive"] [remote=tidb-khkrpa-tidb-0.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:10080] [2025/09/01 03:54:30.645 +00:00] [WARN] [proxy.go:193] ["fail to recv activity from remote, stay inactive and wait to next checking round"] [remote=tidb-khkrpa-tidb-0.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:10080] [interval=2s] [error="dial tcp 10.244.1.145:10080: connect: connection refused"] [2025/09/01 03:54:32.635 +00:00] [WARN] [proxy.go:193] ["fail to recv activity from remote, stay inactive and wait to next checking round"] [remote=tidb-khkrpa-tidb-0.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:10080] [interval=2s] [error="dial tcp 10.244.1.145:10080: connect: connection refused"] [2025/09/01 03:54:52.364 +00:00] [INFO] [grpc_service.go:99] ["watch request"] [key=resource_group/controller] [range-end=resource_group/controlles] [start-revision=1333] [2025/09/01 03:54:52.364 +00:00] [INFO] [grpc_service.go:99] ["watch request"] [key=resource_group/settings] [range-end=resource_group/settingt] [start-revision=1333] [2025/09/01 03:55:40.641 +00:00] [WARN] [tidb.go:65] ["Alive of TiDB has expired, maybe local time in different hosts are not synchronized"] [key=/topology/tidb/tidb-khkrpa-tidb-2.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:4000/ttl] [value=1756698892333122295] [2025/09/01 03:55:45.643 +00:00] [WARN] [tidb.go:65] ["Alive of TiDB has expired, maybe local time in different hosts are not synchronized"] [key=/topology/tidb/tidb-khkrpa-tidb-2.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:4000/ttl] [value=1756698892333122295] [2025/09/01 03:55:50.644 +00:00] [WARN] [tidb.go:65] ["Alive of TiDB has expired, maybe local time in different hosts are not synchronized"] [key=/topology/tidb/tidb-khkrpa-tidb-2.tidb-khkrpa-tidb-headless.ns-shbjn.svc.cluster.local:4000/ttl] [value=1756698892333122295] `kubectl delete pod tidb-khkrpa-mysql-client --force --namespace ns-shbjn ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "tidb-khkrpa-mysql-client" force deleted delete cluster tidb-khkrpa `kbcli cluster delete tidb-khkrpa --auto-approve --namespace ns-shbjn ` Cluster tidb-khkrpa deleted pod_info:tidb-khkrpa-tidb-0 3/3 Running 0 2m7s tidb-khkrpa-tidb-1 3/3 Running 0 2m50s tidb-khkrpa-tidb-pd-0 3/3 Running 0 4m57s tidb-khkrpa-tikv-0 2/2 Running 0 3m7s tidb-khkrpa-tikv-1 2/2 Running 0 3m20s pod_info:tidb-khkrpa-tidb-0 3/3 Terminating 0 2m27s tidb-khkrpa-tidb-1 3/3 Terminating 0 3m10s tidb-khkrpa-tidb-pd-0 3/3 Running 0 5m17s tidb-khkrpa-tikv-0 2/2 Running 0 3m27s tidb-khkrpa-tikv-1 2/2 Running 0 3m40s No resources found in ns-shbjn namespace. delete cluster pod done No resources found in ns-shbjn namespace. check cluster resource non-exist OK: pvc No resources found in ns-shbjn namespace. delete cluster done No resources found in ns-shbjn namespace. No resources found in ns-shbjn namespace. No resources found in ns-shbjn namespace. Tidb Test Suite All Done! --------------------------------------Tidb (Topology = cluster Replicas 2) Test Result-------------------------------------- [PASSED]|[Create]|[Topology=cluster;ComponentVersion=tidb;]|[Description=Create a cluster with the specified topology cluster and component version tidb] [PASSED]|[Connect]|[ComponentName=tidb]|[Description=Connect to the cluster] [PASSED]|[HorizontalScaling Out]|[ComponentName=tikv]|[Description=HorizontalScaling Out the cluster specify component tikv] [PASSED]|[VolumeExpansion]|[ComponentName=tidb-pd,tikv]|[Description=VolumeExpansion the cluster specify component tidb-pd,tikv] [PASSED]|[Stop]|[-]|[Description=Stop the cluster] [PASSED]|[Start]|[-]|[Description=Start the cluster] [PASSED]|[VerticalScaling]|[ComponentName=tidb]|[Description=VerticalScaling the cluster specify component tidb] [PASSED]|[CheckDBClientBatchData]|[OpsType=VerticalScaling]|[Description=Check the cluster batch data consistent] [PASSED]|[VerticalScaling]|[ComponentName=tidb-pd,tikv]|[Description=VerticalScaling the cluster specify component tidb-pd,tikv] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [PASSED]|[Restart]|[ComponentName=tidb,tikv]|[Description=Restart the cluster specify component tidb,tikv] [PASSED]|[HorizontalScaling Out]|[ComponentName=tidb]|[Description=HorizontalScaling Out the cluster specify component tidb] [PASSED]|[HorizontalScaling In]|[ComponentName=tidb]|[Description=HorizontalScaling In the cluster specify component tidb] [PASSED]|[No-Failover]|[HA=Connection Stress;ComponentName=tidb]|[Description=Simulates conditions where pods experience connection stress either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Connection load.] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]