source commons files source engines files source kubeblocks files source kubedb files CLUSTER_NAME: `kubectl get namespace | grep ns-itqls ` `kubectl create namespace ns-itqls` namespace/ns-itqls created create namespace ns-itqls done download kbcli `gh release list --repo apecloud/kbcli --limit 100 | (grep "1.0" || true)` `curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v1.0.1` Your system is linux_amd64 Installing kbcli ... Downloading ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 33.6M 100 33.6M 0 0 164M 0 --:--:-- --:--:-- --:--:-- 164M kbcli installed successfully. Kubernetes: v1.32.6 KubeBlocks: 1.0.1 kbcli: 1.0.1 Make sure your docker service is running and begin your journey with kbcli: kbcli playground init For more information on how to get started, please visit: https://kubeblocks.io download kbcli v1.0.1 done Kubernetes: v1.32.6 KubeBlocks: 1.0.1 kbcli: 1.0.1 Kubernetes Env: v1.32.6 check snapshot controller check snapshot controller done POD_RESOURCES: aks kb-default-sc found aks default-vsc found found default storage class: default KubeBlocks version is:1.0.1 skip upgrade KubeBlocks current KubeBlocks version: 1.0.1 Error: no repositories to show helm repo add chaos-mesh https://charts.chaos-mesh.org "chaos-mesh" has been added to your repositories add helm chart repo chaos-mesh success chaos mesh already installed check component definition check component definition set component name:mysql set component version set component version:apecloud-mysql set service versions:8.0.30 set service versions sorted:8.0.30 set apecloud-mysql component definition set apecloud-mysql component definition apecloud-mysql-1.0.1 REPORT_COUNT 0:0 set replicas first:3,8.0.30 set replicas third:3,8.0.30 set replicas fourth:3,8.0.30 set minimum cmpv service version set minimum cmpv service version replicas:3,8.0.30 REPORT_COUNT:1 CLUSTER_TOPOLOGY:apecloud-mysql topology apecloud-mysql found in cluster definition apecloud-mysql set apecloud-mysql component definition set apecloud-mysql component definition apecloud-mysql-1.0.1 LIMIT_CPU:0.1 LIMIT_MEMORY:0.5 storage size: 1 CLUSTER_NAME:apemysql-zpbadp No resources found in ns-itqls namespace. pod_info: termination_policy:DoNotTerminate create 3 replica DoNotTerminate apecloud-mysql cluster check component definition set component definition by component version check cmpd by labels check cmpd by compDefs set component definition: apecloud-mysql-1.0.1 by component version:apecloud-mysql apiVersion: apps.kubeblocks.io/v1 kind: Cluster metadata: name: apemysql-zpbadp namespace: ns-itqls labels: app.kubernetes.io/instance: apemysql-zpbadp spec: clusterDef: apecloud-mysql topology: apecloud-mysql terminationPolicy: DoNotTerminate componentSpecs: - name: mysql serviceVersion: 8.0.30 disableExporter: true replicas: 3 resources: requests: cpu: 100m memory: 0.5Gi limits: cpu: 100m memory: 0.5Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi `kubectl apply -f test_create_apemysql-zpbadp.yaml` cluster.apps.kubeblocks.io/apemysql-zpbadp created apply test_create_apemysql-zpbadp.yaml Success `rm -rf test_create_apemysql-zpbadp.yaml` check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Creating Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:21 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:22 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:23 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check pod apemysql-zpbadp-mysql-0 container_name mysql exist password L940842zuWMRS68t check pod apemysql-zpbadp-mysql-0 container_name vtablet exist password L940842zuWMRS68t check pod apemysql-zpbadp-mysql-0 container_name kbagent exist password L940842zuWMRS68t check pod apemysql-zpbadp-mysql-0 container_name config-manager exist password L940842zuWMRS68t No container logs contain secret password. describe cluster `kbcli cluster describe apemysql-zpbadp --namespace ns-itqls ` Name: apemysql-zpbadp Created Time: Sep 11,2025 17:21 UTC+0800 NAMESPACE CLUSTER-DEFINITION TOPOLOGY STATUS TERMINATION-POLICY ns-itqls apecloud-mysql apecloud-mysql Running DoNotTerminate Endpoints: COMPONENT INTERNAL EXTERNAL mysql apemysql-zpbadp-mysql.ns-itqls.svc.cluster.local:3306 Topology: COMPONENT SERVICE-VERSION INSTANCE ROLE STATUS AZ NODE CREATED-TIME mysql 8.0.30 apemysql-zpbadp-mysql-0 leader Running 0 aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:21 UTC+0800 mysql 8.0.30 apemysql-zpbadp-mysql-1 follower Running 0 aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:22 UTC+0800 mysql 8.0.30 apemysql-zpbadp-mysql-2 follower Running 0 aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:23 UTC+0800 Resources Allocation: COMPONENT INSTANCE-TEMPLATE CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS mysql 100m / 100m 512Mi / 512Mi data:1Gi default Images: COMPONENT COMPONENT-DEFINITION IMAGE mysql apecloud-mysql-1.0.1 docker.io/apecloud/apecloud-mysql-server:8.0.30-5.beta3.20240330.g94d1caf.15 docker.io/apecloud/apecloud-mysql-scale:0.2.7 docker.io/apecloud/kubeblocks-tools:1.0.1 Data Protection: BACKUP-REPO AUTO-BACKUP BACKUP-SCHEDULE BACKUP-METHOD BACKUP-RETENTION RECOVERABLE-TIME Show cluster events: kbcli cluster list-events -n ns-itqls apemysql-zpbadp `kbcli cluster label apemysql-zpbadp app.kubernetes.io/instance- --namespace ns-itqls ` `kbcli cluster label apemysql-zpbadp app.kubernetes.io/instance=apemysql-zpbadp --namespace ns-itqls ` `kbcli cluster label apemysql-zpbadp --list --namespace ns-itqls ` NAME NAMESPACE LABELS apemysql-zpbadp ns-itqls app.kubernetes.io/instance=apemysql-zpbadp clusterdefinition.kubeblocks.io/name=apecloud-mysql label cluster app.kubernetes.io/instance=apemysql-zpbadp Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=apemysql-zpbadp --namespace ns-itqls ` `kbcli cluster label apemysql-zpbadp --list --namespace ns-itqls ` NAME NAMESPACE LABELS apemysql-zpbadp ns-itqls app.kubernetes.io/instance=apemysql-zpbadp case.name=kbcli.test1 clusterdefinition.kubeblocks.io/name=apecloud-mysql label cluster case.name=kbcli.test1 Success `kbcli cluster label apemysql-zpbadp case.name=kbcli.test2 --overwrite --namespace ns-itqls ` `kbcli cluster label apemysql-zpbadp --list --namespace ns-itqls ` NAME NAMESPACE LABELS apemysql-zpbadp ns-itqls app.kubernetes.io/instance=apemysql-zpbadp case.name=kbcli.test2 clusterdefinition.kubeblocks.io/name=apecloud-mysql label cluster case.name=kbcli.test2 Success `kbcli cluster label apemysql-zpbadp case.name- --namespace ns-itqls ` `kbcli cluster label apemysql-zpbadp --list --namespace ns-itqls ` NAME NAMESPACE LABELS apemysql-zpbadp ns-itqls app.kubernetes.io/instance=apemysql-zpbadp clusterdefinition.kubeblocks.io/name=apecloud-mysql delete cluster label case.name Success cluster connect `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. SERVER_ID IP_PORT ROLE CONNECTED LOG_DELAY_NUM APPLY_DELAY_NUM 1 apemysql-zpbadp-mysql-0.apemysql-zpbadp-mysql-headless:13306 Leader YES 0 0 2 apemysql-zpbadp-mysql-1.apemysql-zpbadp-mysql-headless:13306 Follower YES 0 1 3 apemysql-zpbadp-mysql-2.apemysql-zpbadp-mysql-headless:13306 Follower YES 0 1 `echo "echo \"show variables like 'max_connections';\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Variable_name Value max_connections 83 connect cluster Success set max_connections to 83 insert batch data by db client Error from server (NotFound): pods "test-db-client-executionloop-apemysql-zpbadp" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-apemysql-zpbadp --namespace ns-itqls ` Error from server (NotFound): pods "test-db-client-executionloop-apemysql-zpbadp" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-executionloop-apemysql-zpbadp" not found `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql apiVersion: v1 kind: Pod metadata: name: test-db-client-executionloop-apemysql-zpbadp namespace: ns-itqls spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "apemysql-zpbadp-mysql.ns-itqls.svc.cluster.local" - "--user" - "root" - "--password" - "L940842zuWMRS68t" - "--port" - "3306" - "--dbtype" - "mysql" - "--test" - "executionloop" - "--duration" - "60" - "--interval" - "1" restartPolicy: Never `kubectl apply -f test-db-client-executionloop-apemysql-zpbadp.yaml` pod/test-db-client-executionloop-apemysql-zpbadp created apply test-db-client-executionloop-apemysql-zpbadp.yaml Success `rm -rf test-db-client-executionloop-apemysql-zpbadp.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 1/1 Running 0 5s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 1/1 Running 0 9s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 1/1 Running 0 14s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 1/1 Running 0 19s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 1/1 Running 0 24s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 1/1 Running 0 29s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 1/1 Running 0 35s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 1/1 Running 0 40s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 1/1 Running 0 45s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 1/1 Running 0 50s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 1/1 Running 0 55s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 1/1 Running 0 60s check pod test-db-client-executionloop-apemysql-zpbadp status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-apemysql-zpbadp 0/1 Completed 0 65s check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:21 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:22 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:23 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done CREATE DATABASE IF NOT EXISTS executions_loop; drop table executions_loop_table DROP TABLE IF EXISTS executions_loop.executions_loop_table; create table executions_loop_table CREATE TABLE IF NOT EXISTS executions_loop.executions_loop_table (id INT PRIMARY KEY AUTO_INCREMENT, value VARCHAR(255), tinyint_col TINYINT, smallint_col SMALLINT, mediumint_col MEDIUMINT, int_col INT, bigint_col BIGINT, float_col FLOAT, double_col DOUBLE, decimal_col DECIMAL(10, 2), date_col DATE, time_col TIME, datetime_col DATETIME, timestamp_col TIMESTAMP, year_col YEAR, char_col CHAR(10), text_col TEXT, blob_col BLOB, binary_col BINARY(10), varbinary_col VARBINARY(255), enum_col ENUM('Option1', 'Option2', 'Option3'), set_col SET('Value1', 'Value2', 'Value3') ); Execution loop start:INSERT INTO executions_loop.executions_loop_table (value, tinyint_col, smallint_col, mediumint_col, int_col, bigint_col, float_col, double_col, decimal_col, date_col, time_col, datetime_col, timestamp_col, year_col, char_col, text_col, blob_col, binary_col, varbinary_col, enum_col, set_col) VALUES ('executions_loop_test_1', 20, 20383, 2031179, -592667113, -4069166708276450625, 0.5274399, 0.5027841647238662, 46.19356914203841, '2025-09-11', '09:26:13', '2025-09-11 09:26:13.998', CURRENT_TIMESTAMP, 2123, 'LznSQzNKK3', 'vDazDt4B4cEi8z55yE1ZYZNDbOyvgDxXUZNUVrDdkgyFaFGjdYAnAwaIgM8XflyEX4uBRkF9j6i9XqHYFAyjvZRSwPnDwmfkGncYqfH5F2t7Z6zt3uUDY7gwDiPGhNIFGKhyRS95wq64eqGRbqpDBPqbRwHcJZpfH92q3qu6CYGIA2YC5UCY76jjWtM1M5KnpC2h0z0O46KGJlQc4eGvqt9U3ztW0aueR0IlRW0FIsUXjtsOk3Rc7ktuNbFyhw5', UNHEX(REPLACE('db7ed3d20aa893e180d6', ' ', '')), UNHEX(REPLACE('f02ceae3e2c10a44bda3', ' ', '')), UNHEX(REPLACE('87f8ae40b22edf79885659f18a0bbc01311ce04f15149378ab0e46ed055984a2a09226bcc9e1d7e00208530e76cd60a958268f1ef8c3dacd3eeb69d034acf5bbfe68e405bc426cc26c79aa0e3946371662996c93d0c932869c0bcac9f3aaf0b0bf6591c089775195c7e8f8f05812c4bc410ecb3f0667386fed952c933c54ff7bc6abbcf8a6e9f20a7e84fc3ae6007020e89e070aba639b63fac23062e0ee07413b6b384f7bc0e365db93f8a90d52dca6d2828f763eb11cb3b20eb9c301b016d708c68c4de52b154e8f2fc2028c4510b17cbb4cac98dcceada5f83a5b98b1455a5dc4322bb57e9b282e8a7bf434f9593ee7d04bf4045102ab8c541667f0383d', ' ', '')), 'Option1', 'Value2' ); [ 1s ] executions total: 31 successful: 31 failed: 0 disconnect: 0 [ 2s ] executions total: 77 successful: 77 failed: 0 disconnect: 0 [ 3s ] executions total: 123 successful: 123 failed: 0 disconnect: 0 [ 4s ] executions total: 168 successful: 168 failed: 0 disconnect: 0 [ 5s ] executions total: 219 successful: 219 failed: 0 disconnect: 0 [ 6s ] executions total: 265 successful: 265 failed: 0 disconnect: 0 [ 7s ] executions total: 314 successful: 314 failed: 0 disconnect: 0 [ 8s ] executions total: 361 successful: 361 failed: 0 disconnect: 0 [ 9s ] executions total: 408 successful: 408 failed: 0 disconnect: 0 [ 10s ] executions total: 455 successful: 455 failed: 0 disconnect: 0 [ 11s ] executions total: 502 successful: 502 failed: 0 disconnect: 0 [ 12s ] executions total: 549 successful: 549 failed: 0 disconnect: 0 [ 13s ] executions total: 593 successful: 593 failed: 0 disconnect: 0 [ 14s ] executions total: 641 successful: 641 failed: 0 disconnect: 0 [ 15s ] executions total: 688 successful: 688 failed: 0 disconnect: 0 [ 16s ] executions total: 731 successful: 731 failed: 0 disconnect: 0 java.io.IOException: Failed to execute query: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure The last packet successfully received from the server was 1,168 milliseconds ago. The last packet sent successfully to the server was 1,168 milliseconds ago. [2025-09-11 09:26:31.538] Connection error occurred! [ 17s ] executions total: 748 successful: 747 failed: 1 disconnect: 1 java.io.IOException: Failed to execute query: java.sql.SQLException: The consensus follower is not allowed to to do current operation. [ 18s ] executions total: 749 successful: 747 failed: 2 disconnect: 1 [2025-09-11 09:26:31.538] Connection error occurred! [2025-09-11 09:26:33.575] Connection successfully recovered! The connection was restored in 2037 milliseconds. [ 19s ] executions total: 750 successful: 748 failed: 2 disconnect: 1 [ 20s ] executions total: 796 successful: 794 failed: 2 disconnect: 1 [ 21s ] executions total: 841 successful: 839 failed: 2 disconnect: 1 [ 22s ] executions total: 889 successful: 887 failed: 2 disconnect: 1 [ 23s ] executions total: 938 successful: 936 failed: 2 disconnect: 1 [ 24s ] executions total: 984 successful: 982 failed: 2 disconnect: 1 [ 25s ] executions total: 1029 successful: 1027 failed: 2 disconnect: 1 [ 26s ] executions total: 1075 successful: 1073 failed: 2 disconnect: 1 [ 27s ] executions total: 1122 successful: 1120 failed: 2 disconnect: 1 [ 28s ] executions total: 1173 successful: 1171 failed: 2 disconnect: 1 [ 29s ] executions total: 1222 successful: 1220 failed: 2 disconnect: 1 [ 30s ] executions total: 1268 successful: 1266 failed: 2 disconnect: 1 [ 31s ] executions total: 1315 successful: 1313 failed: 2 disconnect: 1 [ 32s ] executions total: 1366 successful: 1364 failed: 2 disconnect: 1 [ 33s ] executions total: 1413 successful: 1411 failed: 2 disconnect: 1 [ 34s ] executions total: 1463 successful: 1461 failed: 2 disconnect: 1 [ 35s ] executions total: 1515 successful: 1513 failed: 2 disconnect: 1 [ 36s ] executions total: 1562 successful: 1560 failed: 2 disconnect: 1 java.io.IOException: Failed to execute query: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure The last packet successfully received from the server was 1,637 milliseconds ago. The last packet sent successfully to the server was 1,637 milliseconds ago. [2025-09-11 09:26:53.052] Connection error occurred! [ 37s ] executions total: 1594 successful: 1591 failed: 3 disconnect: 2 Failed to connect to MySQL database: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. Trying with database MySQL. [2025-09-11 09:26:53.052] Connection error occurred! [2025-09-11 09:26:56.146] Connection successfully recovered! The connection was restored in 3094 milliseconds. [ 38s ] executions total: 1595 successful: 1592 failed: 3 disconnect: 2 [ 39s ] executions total: 1643 successful: 1640 failed: 3 disconnect: 2 [ 40s ] executions total: 1689 successful: 1686 failed: 3 disconnect: 2 [ 41s ] executions total: 1740 successful: 1737 failed: 3 disconnect: 2 [ 42s ] executions total: 1789 successful: 1786 failed: 3 disconnect: 2 [ 43s ] executions total: 1835 successful: 1832 failed: 3 disconnect: 2 [ 44s ] executions total: 1880 successful: 1877 failed: 3 disconnect: 2 [ 45s ] executions total: 1925 successful: 1922 failed: 3 disconnect: 2 [ 46s ] executions total: 1971 successful: 1968 failed: 3 disconnect: 2 [ 47s ] executions total: 2019 successful: 2016 failed: 3 disconnect: 2 [ 48s ] executions total: 2066 successful: 2063 failed: 3 disconnect: 2 [ 49s ] executions total: 2112 successful: 2109 failed: 3 disconnect: 2 [ 50s ] executions total: 2161 successful: 2158 failed: 3 disconnect: 2 [ 51s ] executions total: 2206 successful: 2203 failed: 3 disconnect: 2 [ 52s ] executions total: 2251 successful: 2248 failed: 3 disconnect: 2 [ 53s ] executions total: 2297 successful: 2294 failed: 3 disconnect: 2 [ 54s ] executions total: 2344 successful: 2341 failed: 3 disconnect: 2 [ 55s ] executions total: 2388 successful: 2385 failed: 3 disconnect: 2 [ 60s ] executions total: 2414 successful: 2411 failed: 3 disconnect: 2 Test Result: Total Executions: 2414 Successful Executions: 2411 Failed Executions: 3 Disconnection Counts: 2 Connection Information: Database Type: mysql Host: apemysql-zpbadp-mysql.ns-itqls.svc.cluster.local Port: 3306 Database: Table: User: root Org: Access Mode: mysql Test Type: executionloop Query: Duration: 60 seconds Interval: 1 seconds DB_CLIENT_BATCH_DATA_COUNT: 2411 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-apemysql-zpbadp --namespace ns-itqls ` pod/test-db-client-executionloop-apemysql-zpbadp patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-executionloop-apemysql-zpbadp" force deleted `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql set db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` set db_client batch data retry times: 1 set db_client batch data retry times: 2 set db_client batch data retry times: 3 set db_client batch data retry times: 4 set db_client batch data retry times: 5 set db_client batch data Failure Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. count(*) 2413 set DB_CLIENT_BATCH_DATA_COUNT: `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"CREATE DATABASE IF NOT EXISTS mydb; use mydb; DROP TABLE IF EXISTS tmp_table; CREATE TABLE IF NOT EXISTS tmp_table (id INT PRIMARY KEY AUTO_INCREMENT, value VARCHAR(255)); INSERT INTO tmp_table (value) VALUES ('byxag');\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. add consistent data byxag Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"CREATE DATABASE IF NOT EXISTS mydb; use mydb; DROP TABLE IF EXISTS tmp_table; CREATE TABLE IF NOT EXISTS tmp_table (id INT PRIMARY KEY AUTO_INCREMENT, value VARCHAR(255)); INSERT INTO tmp_table (value) VALUES ('byxag');\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 7504 (HY000) at line 1: The consensus follower is not allowed to to do current operation. command terminated with exit code 1 check add consistent data readonly Success LB_TYPE is set to: internet cluster expose check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster expose apemysql-zpbadp --auto-approve --force=true --type internet --enable true --components mysql --role-selector leader --namespace ns-itqls ` OpsRequest apemysql-zpbadp-expose-8xgp2 created successfully, you can view the progress: kbcli cluster describe-ops apemysql-zpbadp-expose-8xgp2 -n ns-itqls check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-expose-8xgp2 ns-itqls Expose apemysql-zpbadp Running -/- Sep 11,2025 17:27 UTC+0800 check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:21 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:22 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:23 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-expose-8xgp2 ns-itqls Expose apemysql-zpbadp mysql Running 0/1 Sep 11,2025 17:27 UTC+0800 ops_status:apemysql-zpbadp-expose-8xgp2 ns-itqls Expose apemysql-zpbadp mysql Running 0/1 Sep 11,2025 17:27 UTC+0800 ops_status:apemysql-zpbadp-expose-8xgp2 ns-itqls Expose apemysql-zpbadp mysql Running 0/1 Sep 11,2025 17:27 UTC+0800 ops_status:apemysql-zpbadp-expose-8xgp2 ns-itqls Expose apemysql-zpbadp mysql Running 0/1 Sep 11,2025 17:27 UTC+0800 ops_status:apemysql-zpbadp-expose-8xgp2 ns-itqls Expose apemysql-zpbadp mysql Running 0/1 Sep 11,2025 17:27 UTC+0800 ops_status:apemysql-zpbadp-expose-8xgp2 ns-itqls Expose apemysql-zpbadp mysql Running 0/1 Sep 11,2025 17:27 UTC+0800 check ops status done ops_status:apemysql-zpbadp-expose-8xgp2 ns-itqls Expose apemysql-zpbadp mysql Succeed 1/1 Sep 11,2025 17:27 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations apemysql-zpbadp-expose-8xgp2 --namespace ns-itqls ` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-expose-8xgp2 patched `kbcli cluster delete-ops --name apemysql-zpbadp-expose-8xgp2 --force --auto-approve --namespace ns-itqls ` OpsRequest apemysql-zpbadp-expose-8xgp2 deleted `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success cluster configure component_tmp: mysql apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: apemysql-zpbadp-reconfiguring- namespace: ns-itqls spec: type: Reconfiguring clusterName: apemysql-zpbadp force: true reconfigures: - componentName: mysql parameters: - key: max_connections value: '1000' - key: general_log value: 'OFF' check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_apemysql-zpbadp.yaml` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-reconfiguring-lbg58 created create test_ops_cluster_apemysql-zpbadp.yaml Success `rm -rf test_ops_cluster_apemysql-zpbadp.yaml` check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-reconfiguring-lbg58 ns-itqls Reconfiguring apemysql-zpbadp mysql,mysql Running -/- Sep 11,2025 17:28 UTC+0800 check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:21 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:22 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:23 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-reconfiguring-lbg58 ns-itqls Reconfiguring apemysql-zpbadp mysql,mysql Succeed -/- Sep 11,2025 17:28 UTC+0800 check ops status done ops_status:apemysql-zpbadp-reconfiguring-lbg58 ns-itqls Reconfiguring apemysql-zpbadp mysql,mysql Succeed -/- Sep 11,2025 17:28 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations apemysql-zpbadp-reconfiguring-lbg58 --namespace ns-itqls ` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-reconfiguring-lbg58 patched `kbcli cluster delete-ops --name apemysql-zpbadp-reconfiguring-lbg58 --force --auto-approve --namespace ns-itqls ` OpsRequest apemysql-zpbadp-reconfiguring-lbg58 deleted component_config:mysql check config variables `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. current value_actual: 1000 configure:[max_connections] result actual:[1000] equal expected:[1000] `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. current value_actual: OFF configure:[general_log] result actual:[OFF] equal expected:[OFF] `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success cluster configure component_tmp: mysql apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: apemysql-zpbadp-reconfiguring- namespace: ns-itqls spec: type: Reconfiguring clusterName: apemysql-zpbadp force: true reconfigures: - componentName: mysql parameters: - key: max_connections value: '2000' - key: general_log value: 'ON' - key: innodb_sort_buffer_size value: '2097152' check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_apemysql-zpbadp.yaml` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-reconfiguring-nww6v created create test_ops_cluster_apemysql-zpbadp.yaml Success `rm -rf test_ops_cluster_apemysql-zpbadp.yaml` check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-reconfiguring-nww6v ns-itqls Reconfiguring apemysql-zpbadp mysql,mysql Running -/- Sep 11,2025 17:28 UTC+0800 check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Updating Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:31 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:30 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:29 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash` check cluster connect done check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-reconfiguring-nww6v ns-itqls Reconfiguring apemysql-zpbadp mysql,mysql Succeed -/- Sep 11,2025 17:28 UTC+0800 check ops status done ops_status:apemysql-zpbadp-reconfiguring-nww6v ns-itqls Reconfiguring apemysql-zpbadp mysql,mysql Succeed -/- Sep 11,2025 17:28 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations apemysql-zpbadp-reconfiguring-nww6v --namespace ns-itqls ` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-reconfiguring-nww6v patched `kbcli cluster delete-ops --name apemysql-zpbadp-reconfiguring-nww6v --force --auto-approve --namespace ns-itqls ` OpsRequest apemysql-zpbadp-reconfiguring-nww6v deleted component_config:mysql check config variables `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. current value_actual: 2000 configure:[max_connections] result actual:[2000] equal expected:[2000] `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. current value_actual: ON configure:[general_log] result actual:[ON] equal expected:[ON] `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. current value_actual: 2097152 configure:[innodb_sort_buffer_size] result actual:[2097152] equal expected:[2097152] `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success test failover podkill check cluster status before cluster-failover-podkill check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podkill-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podkill-apemysql-zpbadp" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podkill-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podkill-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-2 mode: all action: pod-kill `kubectl apply -f test-chaos-mesh-podkill-apemysql-zpbadp.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podkill-apemysql-zpbadp created apply test-chaos-mesh-podkill-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-podkill-apemysql-zpbadp.yaml` check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Updating Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:31 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:30 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:31 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podkill-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podkill-apemysql-zpbadp" force deleted podchaos.chaos-mesh.org/test-chaos-mesh-podkill-apemysql-zpbadp patched check failover pod name failover pod name:apemysql-zpbadp-mysql-0 failover podkill Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success cluster configure component_tmp: mysql apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: apemysql-zpbadp-reconfiguring- namespace: ns-itqls spec: type: Reconfiguring clusterName: apemysql-zpbadp force: true reconfigures: - componentName: mysql parameters: - key: log_error_verbosity value: '3' check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_apemysql-zpbadp.yaml` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-reconfiguring-vr8f9 created create test_ops_cluster_apemysql-zpbadp.yaml Success `rm -rf test_ops_cluster_apemysql-zpbadp.yaml` check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-reconfiguring-vr8f9 ns-itqls Reconfiguring apemysql-zpbadp mysql,mysql Creating -/- Sep 11,2025 17:32 UTC+0800 check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:31 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:30 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:31 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-reconfiguring-vr8f9 ns-itqls Reconfiguring apemysql-zpbadp mysql,mysql Succeed -/- Sep 11,2025 17:32 UTC+0800 check ops status done ops_status:apemysql-zpbadp-reconfiguring-vr8f9 ns-itqls Reconfiguring apemysql-zpbadp mysql,mysql Succeed -/- Sep 11,2025 17:32 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations apemysql-zpbadp-reconfiguring-vr8f9 --namespace ns-itqls ` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-reconfiguring-vr8f9 patched `kbcli cluster delete-ops --name apemysql-zpbadp-reconfiguring-vr8f9 --force --auto-approve --namespace ns-itqls ` OpsRequest apemysql-zpbadp-reconfiguring-vr8f9 deleted component_config:mysql check config variables `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. current value_actual: 3 configure:[log_error_verbosity] result actual:[3] equal expected:[3] `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success test switchover cluster promote check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster promote apemysql-zpbadp --auto-approve --force=true --instance apemysql-zpbadp-mysql-0 --candidate apemysql-zpbadp-mysql-1 --namespace ns-itqls ` OpsRequest apemysql-zpbadp-switchover-ksczf created successfully, you can view the progress: kbcli cluster describe-ops apemysql-zpbadp-switchover-ksczf -n ns-itqls check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-switchover-ksczf ns-itqls Switchover apemysql-zpbadp Creating -/- Sep 11,2025 17:32 UTC+0800 check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:31 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:30 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:31 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-1;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash` check cluster connect done check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-switchover-ksczf ns-itqls Switchover apemysql-zpbadp apemysql-zpbadp-mysql Succeed 1/1 Sep 11,2025 17:32 UTC+0800 check ops status done ops_status:apemysql-zpbadp-switchover-ksczf ns-itqls Switchover apemysql-zpbadp apemysql-zpbadp-mysql Succeed 1/1 Sep 11,2025 17:32 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations apemysql-zpbadp-switchover-ksczf --namespace ns-itqls ` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-switchover-ksczf patched `kbcli cluster delete-ops --name apemysql-zpbadp-switchover-ksczf --force --auto-approve --namespace ns-itqls ` OpsRequest apemysql-zpbadp-switchover-ksczf deleted `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success switchover pod:apemysql-zpbadp-mysql-1 switchover success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql apiVersion: v1 kind: Pod metadata: name: benchtest-apemysql-zpbadp namespace: ns-itqls spec: containers: - name: test-sysbench imagePullPolicy: IfNotPresent image: docker.io/apecloud/customsuites:latest env: - name: TYPE value: "2" - name: FLAG value: "0" - name: CONFIGS value: "mode:all,driver:mysql,host:apemysql-zpbadp-mysql.ns-itqls.svc.cluster.local,user:root,password:L940842zuWMRS68t,port:3306,db:benchtest,tables:5,threads:4,times:10,size:1000,type:oltp_read_write" restartPolicy: Never `kubectl apply -f benchtest-apemysql-zpbadp.yaml` pod/benchtest-apemysql-zpbadp created apply benchtest-apemysql-zpbadp.yaml Success check pod status pod_status:NAME READY STATUS RESTARTS AGE benchtest-apemysql-zpbadp 0/1 ContainerCreating 0 0s pod_status:NAME READY STATUS RESTARTS AGE benchtest-apemysql-zpbadp 1/1 Running 0 4s pod_status:NAME READY STATUS RESTARTS AGE benchtest-apemysql-zpbadp 1/1 Running 0 9s pod_status:NAME READY STATUS RESTARTS AGE benchtest-apemysql-zpbadp 1/1 Running 0 14s check pod benchtest-apemysql-zpbadp status done pod_status:NAME READY STATUS RESTARTS AGE benchtest-apemysql-zpbadp 0/1 Completed 0 19s `rm -rf benchtest-apemysql-zpbadp.yaml` `kubectl logs benchtest-apemysql-zpbadp --tail 30 --namespace ns-itqls ` [ 7s ] thds: 4 tps: 20.00 qps: 387.96 (r/w/o: 277.97/69.99/40.00) lat (ms,99%): 404.61 err/s: 0.00 reconn/s: 0.00 [ 8s ] thds: 4 tps: 17.00 qps: 333.96 (r/w/o: 231.97/67.99/34.00) lat (ms,99%): 502.20 err/s: 0.00 reconn/s: 0.00 [ 9s ] thds: 4 tps: 18.00 qps: 381.05 (r/w/o: 265.03/80.01/36.00) lat (ms,99%): 427.07 err/s: 0.00 reconn/s: 0.00 [ 10s ] thds: 4 tps: 16.00 qps: 312.98 (r/w/o: 216.98/64.00/32.00) lat (ms,99%): 493.24 err/s: 0.00 reconn/s: 0.00 SQL statistics: queries performed: read: 2436 write: 693 other: 347 total: 3476 transactions: 173 (16.96 per sec.) queries: 3476 (340.78 per sec.) ignored errors: 1 (0.10 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 10.1983s total number of events: 173 Latency (ms): min: 18.61 avg: 233.64 max: 598.41 99th percentile: 493.24 sum: 40419.42 Threads fairness: events (avg/stddev): 43.2500/1.79 execution time (avg/stddev): 10.1049/0.06 `kubectl delete pod benchtest-apemysql-zpbadp --force --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "benchtest-apemysql-zpbadp" force deleted LB_TYPE is set to: internet No resources found in ns-itqls namespace. `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql apiVersion: v1 kind: Pod metadata: name: benchtest-apemysql-zpbadp namespace: ns-itqls spec: containers: - name: test-sysbench imagePullPolicy: IfNotPresent image: docker.io/apecloud/customsuites:latest env: - name: TYPE value: "2" - name: FLAG value: "0" - name: CONFIGS value: "mode:all,driver:mysql,host:134.33.160.28,user:root,password:L940842zuWMRS68t,port:3306,db:benchtest,tables:5,threads:4,times:10,size:1000,type:oltp_read_write" restartPolicy: Never `kubectl apply -f benchtest-apemysql-zpbadp.yaml` pod/benchtest-apemysql-zpbadp created apply benchtest-apemysql-zpbadp.yaml Success check pod status pod_status:NAME READY STATUS RESTARTS AGE benchtest-apemysql-zpbadp 0/1 ContainerCreating 0 0s pod_status:NAME READY STATUS RESTARTS AGE benchtest-apemysql-zpbadp 1/1 Running 0 4s pod_status:NAME READY STATUS RESTARTS AGE benchtest-apemysql-zpbadp 1/1 Running 0 9s pod_status:NAME READY STATUS RESTARTS AGE benchtest-apemysql-zpbadp 1/1 Running 0 14s pod_status:NAME READY STATUS RESTARTS AGE benchtest-apemysql-zpbadp 1/1 Running 0 19s check pod benchtest-apemysql-zpbadp status done pod_status:NAME READY STATUS RESTARTS AGE benchtest-apemysql-zpbadp 0/1 Completed 0 24s `rm -rf benchtest-apemysql-zpbadp.yaml` `kubectl logs benchtest-apemysql-zpbadp --tail 30 --namespace ns-itqls ` [ 7s ] thds: 4 tps: 17.00 qps: 337.01 (r/w/o: 238.01/65.00/34.00) lat (ms,99%): 484.44 err/s: 0.00 reconn/s: 0.00 [ 8s ] thds: 4 tps: 18.00 qps: 340.00 (r/w/o: 237.00/68.00/35.00) lat (ms,99%): 390.30 err/s: 0.00 reconn/s: 0.00 [ 9s ] thds: 4 tps: 16.00 qps: 344.00 (r/w/o: 239.00/72.00/33.00) lat (ms,99%): 397.39 err/s: 0.00 reconn/s: 0.00 [ 10s ] thds: 4 tps: 16.99 qps: 335.76 (r/w/o: 237.83/63.95/33.98) lat (ms,99%): 502.20 err/s: 0.00 reconn/s: 0.00 SQL statistics: queries performed: read: 2422 write: 692 other: 346 total: 3460 transactions: 173 (17.13 per sec.) queries: 3460 (342.51 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 10.1001s total number of events: 173 Latency (ms): min: 22.58 avg: 232.02 max: 501.07 99th percentile: 484.44 sum: 40138.96 Threads fairness: events (avg/stddev): 43.2500/1.48 execution time (avg/stddev): 10.0347/0.04 `kubectl delete pod benchtest-apemysql-zpbadp --force --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "benchtest-apemysql-zpbadp" force deleted test failover networkcorruptover check cluster status before cluster-failover-networkcorruptover check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-apemysql-zpbadp" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorruptover-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-1 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorruptover-apemysql-zpbadp.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-apemysql-zpbadp created apply test-chaos-mesh-networkcorruptover-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-networkcorruptover-apemysql-zpbadp.yaml` networkcorruptover chaos test waiting 120 seconds check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:31 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:30 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:1Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:31 UTC+0800 check pod status done check cluster role leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 check cluster role done leader: apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-apemysql-zpbadp patched check failover pod name networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-apemysql-zpbadp" force deleted failover pod name:apemysql-zpbadp-mysql-2 failover networkcorruptover Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get pvc -l app.kubernetes.io/instance=apemysql-zpbadp,apps.kubeblocks.io/component-name=mysql,apps.kubeblocks.io/vct-name=data --namespace ns-itqls ` cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in apemysql-zpbadp namespace. `kbcli cluster volume-expand apemysql-zpbadp --auto-approve --force=true --components mysql --volume-claim-templates data --storage 2Gi --namespace ns-itqls ` OpsRequest apemysql-zpbadp-volumeexpansion-m2mfq created successfully, you can view the progress: kbcli cluster describe-ops apemysql-zpbadp-volumeexpansion-m2mfq -n ns-itqls check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-volumeexpansion-m2mfq ns-itqls VolumeExpansion apemysql-zpbadp mysql Creating -/- Sep 11,2025 17:38 UTC+0800 check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Updating Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:31 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:30 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:31 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash` check cluster connect done No resources found in apemysql-zpbadp namespace. check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-volumeexpansion-m2mfq ns-itqls VolumeExpansion apemysql-zpbadp mysql Succeed 3/3 Sep 11,2025 17:38 UTC+0800 check ops status done ops_status:apemysql-zpbadp-volumeexpansion-m2mfq ns-itqls VolumeExpansion apemysql-zpbadp mysql Succeed 3/3 Sep 11,2025 17:38 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations apemysql-zpbadp-volumeexpansion-m2mfq --namespace ns-itqls ` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-volumeexpansion-m2mfq patched `kbcli cluster delete-ops --name apemysql-zpbadp-volumeexpansion-m2mfq --force --auto-approve --namespace ns-itqls ` OpsRequest apemysql-zpbadp-volumeexpansion-m2mfq deleted `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success test failover dnsrandom check cluster status before cluster-failover-dnsrandom check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-apemysql-zpbadp" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnsrandom-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-2 mode: all action: random duration: 2m `kubectl apply -f test-chaos-mesh-dnsrandom-apemysql-zpbadp.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnsrandom-apemysql-zpbadp created apply test-chaos-mesh-dnsrandom-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-dnsrandom-apemysql-zpbadp.yaml` dnsrandom chaos test waiting 120 seconds check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:31 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:30 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:31 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-apemysql-zpbadp" force deleted dnschaos.chaos-mesh.org/test-chaos-mesh-dnsrandom-apemysql-zpbadp patched check failover pod name failover pod name:apemysql-zpbadp-mysql-2 failover dnsrandom Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success cluster stop check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster stop apemysql-zpbadp --auto-approve --force=true --namespace ns-itqls ` OpsRequest apemysql-zpbadp-stop-2qrjl created successfully, you can view the progress: kbcli cluster describe-ops apemysql-zpbadp-stop-2qrjl -n ns-itqls check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-stop-2qrjl ns-itqls Stop apemysql-zpbadp mysql Running 0/3 Sep 11,2025 17:46 UTC+0800 check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Stopping Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping check cluster status done cluster_status:Stopped check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME check pod status done check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-stop-2qrjl ns-itqls Stop apemysql-zpbadp mysql Succeed 3/3 Sep 11,2025 17:46 UTC+0800 check ops status done ops_status:apemysql-zpbadp-stop-2qrjl ns-itqls Stop apemysql-zpbadp mysql Succeed 3/3 Sep 11,2025 17:46 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations apemysql-zpbadp-stop-2qrjl --namespace ns-itqls ` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-stop-2qrjl patched `kbcli cluster delete-ops --name apemysql-zpbadp-stop-2qrjl --force --auto-approve --namespace ns-itqls ` OpsRequest apemysql-zpbadp-stop-2qrjl deleted cluster start check cluster status before ops check cluster status done cluster_status:Stopped `kbcli cluster start apemysql-zpbadp --force=true --namespace ns-itqls ` OpsRequest apemysql-zpbadp-start-99gm2 created successfully, you can view the progress: kbcli cluster describe-ops apemysql-zpbadp-start-99gm2 -n ns-itqls check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-start-99gm2 ns-itqls Start apemysql-zpbadp Running -/- Sep 11,2025 17:48 UTC+0800 check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Updating Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:49 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-start-99gm2 ns-itqls Start apemysql-zpbadp mysql Succeed 3/3 Sep 11,2025 17:48 UTC+0800 check ops status done ops_status:apemysql-zpbadp-start-99gm2 ns-itqls Start apemysql-zpbadp mysql Succeed 3/3 Sep 11,2025 17:48 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations apemysql-zpbadp-start-99gm2 --namespace ns-itqls ` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-start-99gm2 patched `kbcli cluster delete-ops --name apemysql-zpbadp-start-99gm2 --force --auto-approve --namespace ns-itqls ` OpsRequest apemysql-zpbadp-start-99gm2 deleted `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success test failover connectionstress check cluster status before cluster-failover-connectionstress check cluster status done cluster_status:Running Error from server (NotFound): pods "test-db-client-connectionstress-apemysql-zpbadp" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstress-apemysql-zpbadp --namespace ns-itqls ` Error from server (NotFound): pods "test-db-client-connectionstress-apemysql-zpbadp" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-connectionstress-apemysql-zpbadp" not found `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql apiVersion: v1 kind: Pod metadata: name: test-db-client-connectionstress-apemysql-zpbadp namespace: ns-itqls spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "apemysql-zpbadp-mysql.ns-itqls.svc.cluster.local" - "--user" - "root" - "--password" - "L940842zuWMRS68t" - "--port" - "3306" - "--database" - "mysql" - "--dbtype" - "mysql" - "--test" - "connectionstress" - "--connections" - "83" - "--duration" - "60" restartPolicy: Never `kubectl apply -f test-db-client-connectionstress-apemysql-zpbadp.yaml` pod/test-db-client-connectionstress-apemysql-zpbadp created apply test-db-client-connectionstress-apemysql-zpbadp.yaml Success `rm -rf test-db-client-connectionstress-apemysql-zpbadp.yaml` check pod status check pod test-db-client-connectionstress-apemysql-zpbadp status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-apemysql-zpbadp 0/1 Completed 0 5s check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:49 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done --host apemysql-zpbadp-mysql.ns-itqls.svc.cluster.local --user root --password L940842zuWMRS68t --port 3306 --database mysql --dbtype mysql --test connectionstress --connections 83 --duration 60 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider] Test Result: null Connection Information: Database Type: mysql Host: apemysql-zpbadp-mysql.ns-itqls.svc.cluster.local Port: 3306 Database: mysql Table: User: root Org: Access Mode: mysql Test Type: connectionstress Connection Count: 83 Duration: 60 seconds `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstress-apemysql-zpbadp --namespace ns-itqls ` pod/test-db-client-connectionstress-apemysql-zpbadp patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-connectionstress-apemysql-zpbadp" force deleted `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql set db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` set db_client batch data retry times: 1 set db_client batch data retry times: 2 set db_client batch data retry times: 3 set db_client batch data retry times: 4 set db_client batch data retry times: 5 set db_client batch data Failure Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. count(*) 2413 set DB_CLIENT_BATCH_DATA_COUNT: 2413 check failover pod name failover pod name:apemysql-zpbadp-mysql-0 failover connectionstress Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly db_client batch data Success cluster does not need to check monitor currently check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:49 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done test failover networklossover check cluster status before cluster-failover-networklossover check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-apemysql-zpbadp" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-0 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-apemysql-zpbadp.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-apemysql-zpbadp created apply test-chaos-mesh-networklossover-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-networklossover-apemysql-zpbadp.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:49 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-1;follower: apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-apemysql-zpbadp" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-apemysql-zpbadp patched check failover pod name failover pod name:apemysql-zpbadp-mysql-1 failover networklossover Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly db_client batch data Success test failover podfailure check cluster status before cluster-failover-podfailure check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-apemysql-zpbadp" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podfailure-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-1 mode: all action: pod-failure duration: 2m `kubectl apply -f test-chaos-mesh-podfailure-apemysql-zpbadp.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podfailure-apemysql-zpbadp created apply test-chaos-mesh-podfailure-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-podfailure-apemysql-zpbadp.yaml` podfailure chaos test waiting 120 seconds check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Failed Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql cluster_status:Failed cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:49 UTC+0800 check pod status done check cluster role leader: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-1;follower: apemysql-zpbadp-mysql-2 leader: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-1;follower: apemysql-zpbadp-mysql-2 check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-apemysql-zpbadp" force deleted podchaos.chaos-mesh.org/test-chaos-mesh-podfailure-apemysql-zpbadp patched check failover pod name failover pod name:apemysql-zpbadp-mysql-0 failover podfailure Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly db_client batch data Success test failover dnserror check cluster status before cluster-failover-dnserror check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-apemysql-zpbadp" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnserror-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-0 mode: all action: error duration: 2m `kubectl apply -f test-chaos-mesh-dnserror-apemysql-zpbadp.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnserror-apemysql-zpbadp created apply test-chaos-mesh-dnserror-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-dnserror-apemysql-zpbadp.yaml` dnserror chaos test waiting 120 seconds check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:49 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-apemysql-zpbadp" force deleted dnschaos.chaos-mesh.org/test-chaos-mesh-dnserror-apemysql-zpbadp patched check failover pod name failover pod name:apemysql-zpbadp-mysql-0 failover dnserror Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly db_client batch data Success test failover fullcpuover check cluster status before cluster-failover-fullcpuover check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpuover-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpuover-apemysql-zpbadp" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpuover-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-fullcpuover-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-0 mode: all stressors: cpu: workers: 100 load: 100 duration: 2m `kubectl apply -f test-chaos-mesh-fullcpuover-apemysql-zpbadp.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-fullcpuover-apemysql-zpbadp created apply test-chaos-mesh-fullcpuover-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-fullcpuover-apemysql-zpbadp.yaml` fullcpuover chaos test waiting 120 seconds check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:49 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-1;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpuover-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpuover-apemysql-zpbadp" force deleted stresschaos.chaos-mesh.org/test-chaos-mesh-fullcpuover-apemysql-zpbadp patched check failover pod name failover pod name:apemysql-zpbadp-mysql-1 failover fullcpuover Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly db_client batch data Success test failover networkpartition check cluster status before cluster-failover-networkpartition check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-apemysql-zpbadp" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkpartition-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-1 action: partition mode: all target: mode: all selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-0 direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkpartition-apemysql-zpbadp.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkpartition-apemysql-zpbadp created apply test-chaos-mesh-networkpartition-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-networkpartition-apemysql-zpbadp.yaml` networkpartition chaos test waiting 120 seconds check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:49 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-1;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-apemysql-zpbadp" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networkpartition-apemysql-zpbadp patched check failover pod name failover pod name:apemysql-zpbadp-mysql-1 failover networkpartition Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly db_client batch data Success test failover oom check cluster status before cluster-failover-oom check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-apemysql-zpbadp" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-1 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-apemysql-zpbadp.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-apemysql-zpbadp created apply test-chaos-mesh-oom-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-oom-apemysql-zpbadp.yaml` check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:49 UTC+0800 check pod status done check cluster role leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-1 check cluster role done leader: apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-apemysql-zpbadp" force deleted stresschaos.chaos-mesh.org/test-chaos-mesh-oom-apemysql-zpbadp patched check failover pod name failover pod name:apemysql-zpbadp-mysql-2 failover oom Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly db_client batch data Success test failover networkdelayover check cluster status before cluster-failover-networkdelayover check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelayover-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelayover-apemysql-zpbadp" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelayover-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkdelayover-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-2 mode: all action: delay delay: latency: 2000ms correlation: '100' jitter: 0ms direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkdelayover-apemysql-zpbadp.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkdelayover-apemysql-zpbadp created apply test-chaos-mesh-networkdelayover-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-networkdelayover-apemysql-zpbadp.yaml` networkdelayover chaos test waiting 120 seconds check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Updating Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 17:48 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 17:49 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-1;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelayover-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelayover-apemysql-zpbadp" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelayover-apemysql-zpbadp" not found check failover pod name failover pod name:apemysql-zpbadp-mysql-1 failover networkdelayover Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly db_client batch data Success skip apecloud-mysql HorizontalScaling cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart apemysql-zpbadp --auto-approve --force=true --namespace ns-itqls ` OpsRequest apemysql-zpbadp-restart-s6b74 created successfully, you can view the progress: kbcli cluster describe-ops apemysql-zpbadp-restart-s6b74 -n ns-itqls check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-restart-s6b74 ns-itqls Restart apemysql-zpbadp mysql Creating -/- Sep 11,2025 18:04 UTC+0800 check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Updating Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 18:06 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 18:07 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 18:05 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-restart-s6b74 ns-itqls Restart apemysql-zpbadp mysql Succeed 3/3 Sep 11,2025 18:04 UTC+0800 check ops status done ops_status:apemysql-zpbadp-restart-s6b74 ns-itqls Restart apemysql-zpbadp mysql Succeed 3/3 Sep 11,2025 18:04 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations apemysql-zpbadp-restart-s6b74 --namespace ns-itqls ` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-restart-s6b74 patched `kbcli cluster delete-ops --name apemysql-zpbadp-restart-s6b74 --force --auto-approve --namespace ns-itqls ` OpsRequest apemysql-zpbadp-restart-s6b74 deleted `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly db_client batch data Success test failover networkduplicate check cluster status before cluster-failover-networkduplicate check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-apemysql-zpbadp" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkduplicate-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-0 mode: all action: duplicate duplicate: duplicate: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkduplicate-apemysql-zpbadp.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkduplicate-apemysql-zpbadp created apply test-chaos-mesh-networkduplicate-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-networkduplicate-apemysql-zpbadp.yaml` networkduplicate chaos test waiting 120 seconds check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 18:06 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 18:07 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 18:05 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-0;follower: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-apemysql-zpbadp" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networkduplicate-apemysql-zpbadp patched check failover pod name failover pod name:apemysql-zpbadp-mysql-0 failover networkduplicate Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check readonly db_client batch data Success skip cluster Upgrade test failover networkbandwidthover check cluster status before cluster-failover-networkbandwidthover check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-apemysql-zpbadp" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkbandwidthover-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-0 action: bandwidth mode: all bandwidth: rate: '1bps' limit: 20971520 buffer: 10000 duration: 2m `kubectl apply -f test-chaos-mesh-networkbandwidthover-apemysql-zpbadp.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkbandwidthover-apemysql-zpbadp created apply test-chaos-mesh-networkbandwidthover-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-networkbandwidthover-apemysql-zpbadp.yaml` networkbandwidthover chaos test waiting 120 seconds check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 18:06 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 18:07 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 18:05 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-1;follower: apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-apemysql-zpbadp" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networkbandwidthover-apemysql-zpbadp patched check failover pod name failover pod name:apemysql-zpbadp-mysql-1 failover networkbandwidthover Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly db_client batch data Success test failover kill1 check cluster status before cluster-failover-kill1 check cluster status done cluster_status:Running `kill 1` Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file exec return message: check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 18:06 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 18:07 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running leader 0 100m / 100m 512Mi / 512Mi data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 18:05 UTC+0800 check pod status done check cluster role leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 leader: apemysql-zpbadp-mysql-1 apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 check cluster role done leader: apemysql-zpbadp-mysql-2;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash` check cluster connect done check failover pod name failover pod name:apemysql-zpbadp-mysql-2 failover kill1 Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-2 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly db_client batch data Success check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale apemysql-zpbadp --auto-approve --force=true --components mysql --cpu 200m --memory 0.6Gi --namespace ns-itqls ` OpsRequest apemysql-zpbadp-verticalscaling-7h89z created successfully, you can view the progress: kbcli cluster describe-ops apemysql-zpbadp-verticalscaling-7h89z -n ns-itqls check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-verticalscaling-7h89z ns-itqls VerticalScaling apemysql-zpbadp mysql Creating -/- Sep 11,2025 18:12 UTC+0800 check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Updating Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 18:14 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 18:13 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 18:14 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-1;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash` check cluster connect done check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-verticalscaling-7h89z ns-itqls VerticalScaling apemysql-zpbadp mysql Succeed 3/3 Sep 11,2025 18:12 UTC+0800 check ops status done ops_status:apemysql-zpbadp-verticalscaling-7h89z ns-itqls VerticalScaling apemysql-zpbadp mysql Succeed 3/3 Sep 11,2025 18:12 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations apemysql-zpbadp-verticalscaling-7h89z --namespace ns-itqls ` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-verticalscaling-7h89z patched `kbcli cluster delete-ops --name apemysql-zpbadp-verticalscaling-7h89z --force --auto-approve --namespace ns-itqls ` OpsRequest apemysql-zpbadp-verticalscaling-7h89z deleted `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly db_client batch data Success test failover timeoffset check cluster status before cluster-failover-timeoffset check cluster status done cluster_status:Running `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-apemysql-zpbadp" not found Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-apemysql-zpbadp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: TimeChaos metadata: name: test-chaos-mesh-timeoffset-apemysql-zpbadp namespace: ns-itqls spec: selector: namespaces: - ns-itqls labelSelectors: apps.kubeblocks.io/pod-name: apemysql-zpbadp-mysql-1 mode: all timeOffset: '-10m' clockIds: - CLOCK_REALTIME duration: 2m `kubectl apply -f test-chaos-mesh-timeoffset-apemysql-zpbadp.yaml` timechaos.chaos-mesh.org/test-chaos-mesh-timeoffset-apemysql-zpbadp created apply test-chaos-mesh-timeoffset-apemysql-zpbadp.yaml Success `rm -rf test-chaos-mesh-timeoffset-apemysql-zpbadp.yaml` timeoffset chaos test waiting 120 seconds check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql DoNotTerminate Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 18:14 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 18:13 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 18:14 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-1;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-apemysql-zpbadp --namespace ns-itqls ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-apemysql-zpbadp" force deleted timechaos.chaos-mesh.org/test-chaos-mesh-timeoffset-apemysql-zpbadp patched check failover pod name failover pod name:apemysql-zpbadp-mysql-1 failover timeoffset Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly db_client batch data Success cluster update terminationPolicy WipeOut `kbcli cluster update apemysql-zpbadp --termination-policy=WipeOut --namespace ns-itqls ` cluster.apps.kubeblocks.io/apemysql-zpbadp updated check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql WipeOut Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 18:14 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 18:13 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 18:14 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-1;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash` check cluster connect done check cluster storage class cluster volume-snapshot backup `kbcli cluster backup apemysql-zpbadp --method volume-snapshot --namespace ns-itqls ` Backup backup-ns-itqls-apemysql-zpbadp-20250911181750 created successfully, you can view the progress: kbcli cluster list-backups --names=backup-ns-itqls-apemysql-zpbadp-20250911181750 -n ns-itqls check backup status `kbcli cluster list-backups apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION DELETION-POLICY CREATE-TIME COMPLETION-TIME EXPIRATION backup-ns-itqls-apemysql-zpbadp-20250911181750 ns-itqls apemysql-zpbadp volume-snapshot Delete Sep 11,2025 18:17 UTC+0800 backup_status:apemysql-zpbadp-volume-snapshot-Delete backup_status:apemysql-zpbadp-volume-snapshot-Running backup_status:apemysql-zpbadp-volume-snapshot-Running check backup status done backup_status:backup-ns-itqls-apemysql-zpbadp-20250911181750 ns-itqls apemysql-zpbadp volume-snapshot Completed 2Gi 12s Delete Sep 11,2025 18:17 UTC+0800 Sep 11,2025 18:18 UTC+0800 cluster restore backup Error from server (NotFound): opsrequests.operations.kubeblocks.io "apemysql-zpbadp-backup" not found `kbcli cluster describe-backup --names backup-ns-itqls-apemysql-zpbadp-20250911181750 --namespace ns-itqls ` Name: backup-ns-itqls-apemysql-zpbadp-20250911181750 Cluster: apemysql-zpbadp Namespace: ns-itqls Spec: Method: volume-snapshot Policy Name: apemysql-zpbadp-mysql-backup-policy Actions: createVolumeSnapshot-0: panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x2b40d9f] goroutine 1 [running]: github.com/apecloud/kbcli/pkg/cmd/dataprotection.PrintBackupObjDescribe(0xc001338780, 0xc0007dcb08) /home/runner/work/kbcli/kbcli/pkg/cmd/dataprotection/backup.go:480 +0x4bf github.com/apecloud/kbcli/pkg/cmd/dataprotection.DescribeBackups(0xc001338780, ***0xc000c2aec0?, 0x192f1db?, 0xc000d4ed88?***) /home/runner/work/kbcli/kbcli/pkg/cmd/dataprotection/backup.go:458 +0x125 github.com/apecloud/kbcli/pkg/cmd/cluster.describeBackups(0x0?, ***0xc0007d3740?, 0x0?, 0x424719700000000?***) /home/runner/work/kbcli/kbcli/pkg/cmd/cluster/dataprotection.go:204 +0x66 github.com/apecloud/kbcli/pkg/cmd/cluster.NewDescribeBackupCmd.func1(0xc000ac0f08?, ***0xc0007d3740, 0x0, 0x4***) /home/runner/work/kbcli/kbcli/pkg/cmd/cluster/dataprotection.go:195 +0xe5 github.com/spf13/cobra.(*Command).execute(0xc000ac0f08, ***0xc0007d3700, 0x4, 0x4***) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:989 +0xa91 github.com/spf13/cobra.(*Command).ExecuteC(0xc000a1a008) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1117 +0x3ff github.com/spf13/cobra.(*Command).Execute(...) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1041 k8s.io/component-base/cli.run(0xc000a1a008) /home/runner/go/pkg/mod/k8s.io/component-base@v0.29.2/cli/run.go:146 +0x290 k8s.io/component-base/cli.RunNoErrOutput(...) /home/runner/go/pkg/mod/k8s.io/component-base@v0.29.2/cli/run.go:84 main.main() /home/runner/work/kbcli/kbcli/cmd/cli/main.go:31 +0x18 `kbcli cluster restore apemysql-zpbadp-backup --backup backup-ns-itqls-apemysql-zpbadp-20250911181750 --namespace ns-itqls ` Cluster apemysql-zpbadp-backup created check cluster status `kbcli cluster list apemysql-zpbadp-backup --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp-backup ns-itqls apecloud-mysql WipeOut Creating Sep 11,2025 18:18 UTC+0800 clusterdefinition.kubeblocks.io/name=apecloud-mysql cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp-backup --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-backup-mysql-0 ns-itqls apemysql-zpbadp-backup mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 18:18 UTC+0800 apemysql-zpbadp-backup-mysql-1 ns-itqls apemysql-zpbadp-backup mysql Running leader 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 18:18 UTC+0800 apemysql-zpbadp-backup-mysql-2 ns-itqls apemysql-zpbadp-backup mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 18:22 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-backup-mysql-1;follower: apemysql-zpbadp-backup-mysql-0 apemysql-zpbadp-backup-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp-backup` set secret: apemysql-zpbadp-backup-mysql-account-root `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-backup-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-backup-mysql-1 --namespace ns-itqls -- bash` check cluster connect done `kbcli cluster describe-backup --names backup-ns-itqls-apemysql-zpbadp-20250911181750 --namespace ns-itqls ` Name: backup-ns-itqls-apemysql-zpbadp-20250911181750 Cluster: apemysql-zpbadp Namespace: ns-itqls Spec: Method: volume-snapshot Policy Name: apemysql-zpbadp-mysql-backup-policy Actions: createVolumeSnapshot-0: panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x2b40d9f] goroutine 1 [running]: github.com/apecloud/kbcli/pkg/cmd/dataprotection.PrintBackupObjDescribe(0xc001246480, 0xc000684848) /home/runner/work/kbcli/kbcli/pkg/cmd/dataprotection/backup.go:480 +0x4bf github.com/apecloud/kbcli/pkg/cmd/dataprotection.DescribeBackups(0xc001246480, ***0xc0015bd960?, 0x192f1db?, 0xc00160a008?***) /home/runner/work/kbcli/kbcli/pkg/cmd/dataprotection/backup.go:458 +0x125 github.com/apecloud/kbcli/pkg/cmd/cluster.describeBackups(0x0?, ***0xc0007a39c0?, 0x0?, 0x435676a200000000?***) /home/runner/work/kbcli/kbcli/pkg/cmd/cluster/dataprotection.go:204 +0x66 github.com/apecloud/kbcli/pkg/cmd/cluster.NewDescribeBackupCmd.func1(0xc000f44008?, ***0xc0007a39c0, 0x0, 0x4***) /home/runner/work/kbcli/kbcli/pkg/cmd/cluster/dataprotection.go:195 +0xe5 github.com/spf13/cobra.(*Command).execute(0xc000f44008, ***0xc0007a3800, 0x4, 0x4***) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:989 +0xa91 github.com/spf13/cobra.(*Command).ExecuteC(0xc00029b208) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1117 +0x3ff github.com/spf13/cobra.(*Command).Execute(...) /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1041 k8s.io/component-base/cli.run(0xc00029b208) /home/runner/go/pkg/mod/k8s.io/component-base@v0.29.2/cli/run.go:146 +0x290 k8s.io/component-base/cli.RunNoErrOutput(...) /home/runner/go/pkg/mod/k8s.io/component-base@v0.29.2/cli/run.go:84 main.main() /home/runner/work/kbcli/kbcli/cmd/cli/main.go:31 +0x18 cluster connect `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp-backup` set secret: apemysql-zpbadp-backup-mysql-account-root `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. SERVER_ID IP_PORT ROLE CONNECTED LOG_DELAY_NUM APPLY_DELAY_NUM 1 apemysql-zpbadp-backup-mysql-0.apemysql-zpbadp-backup-mysql-headless:13306 Follower YES 0 0 2 apemysql-zpbadp-backup-mysql-1.apemysql-zpbadp-backup-mysql-headless:13306 Leader YES 0 0 3 apemysql-zpbadp-backup-mysql-2.apemysql-zpbadp-backup-mysql-headless:13306 Follower YES 0 0 `echo "echo \"show variables like 'max_connections';\" | mysql -P3306 -hapemysql-zpbadp-backup-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-backup-mysql-1 --namespace ns-itqls -- bash ` Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Variable_name Value max_connections 100 connect cluster Success set max_connections to 100 delete cluster apemysql-zpbadp-backup `kbcli cluster delete apemysql-zpbadp-backup --auto-approve --namespace ns-itqls ` Cluster apemysql-zpbadp-backup deleted pod_info:apemysql-zpbadp-backup-mysql-0 4/4 Running 1 (88s ago) 4m58s apemysql-zpbadp-backup-mysql-1 4/4 Running 0 4m45s apemysql-zpbadp-backup-mysql-2 4/4 Running 0 55s pod_info:apemysql-zpbadp-backup-mysql-0 4/4 Terminating 1 (108s ago) 5m18s apemysql-zpbadp-backup-mysql-1 4/4 Terminating 0 5m5s apemysql-zpbadp-backup-mysql-2 4/4 Terminating 0 75s No resources found in ns-itqls namespace. delete cluster pod done No resources found in ns-itqls namespace. check cluster resource non-exist OK: pvc No resources found in ns-itqls namespace. delete cluster done No resources found in ns-itqls namespace. No resources found in ns-itqls namespace. No resources found in ns-itqls namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups backup-ns-itqls-apemysql-zpbadp-20250911181750 --namespace ns-itqls ` backup.dataprotection.kubeblocks.io/backup-ns-itqls-apemysql-zpbadp-20250911181750 patched `kbcli cluster delete-backup apemysql-zpbadp --name backup-ns-itqls-apemysql-zpbadp-20250911181750 --force --auto-approve --namespace ns-itqls ` Backup backup-ns-itqls-apemysql-zpbadp-20250911181750 deleted No opsrequests found in ns-itqls namespace. cluster xtrabackup backup `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.name***"` `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.namespace***"` `kubectl get secrets kb-backuprepo-fc6xr -n kb-heauh -o jsonpath="***.data.accessKeyId***"` `kubectl get secrets kb-backuprepo-fc6xr -n kb-heauh -o jsonpath="***.data.secretAccessKey***"` KUBEBLOCKS NAMESPACE:kb-heauh get kubeblocks namespace done `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-heauh -o jsonpath="***.items[0].data.root-user***"` `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-heauh -o jsonpath="***.items[0].data.root-password***"` minio_user:kbclitest,minio_password:kbclitest,minio_endpoint:kbcli-test-minio.kb-heauh.svc.cluster.local:9000 list minio bucket kbcli-test `echo 'mc config host add minioserver http://kbcli-test-minio.kb-heauh.svc.cluster.local:9000 kbclitest kbclitest;mc ls minioserver' | kubectl exec -it kbcli-test-minio-8f45f86b6-jvj74 --namespace kb-heauh -- bash` Unable to use a TTY - input is not a terminal or the right kind of file list minio bucket done default backuprepo:backuprepo-kbcli-test exists `kbcli cluster backup apemysql-zpbadp --method xtrabackup --namespace ns-itqls ` Backup backup-ns-itqls-apemysql-zpbadp-20250911182429 created successfully, you can view the progress: kbcli cluster list-backups --names=backup-ns-itqls-apemysql-zpbadp-20250911182429 -n ns-itqls check backup status `kbcli cluster list-backups apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION DELETION-POLICY CREATE-TIME COMPLETION-TIME EXPIRATION backup-ns-itqls-apemysql-zpbadp-20250911182429 ns-itqls apemysql-zpbadp xtrabackup Delete Sep 11,2025 18:24 UTC+0800 backup_status:apemysql-zpbadp-xtrabackup-Running backup_status:apemysql-zpbadp-xtrabackup-Running backup_status:apemysql-zpbadp-xtrabackup-Running check backup status done backup_status:backup-ns-itqls-apemysql-zpbadp-20250911182429 ns-itqls apemysql-zpbadp xtrabackup Completed 9494338 11s Delete Sep 11,2025 18:24 UTC+0800 Sep 11,2025 18:24 UTC+0800 cluster restore backup Error from server (NotFound): opsrequests.operations.kubeblocks.io "apemysql-zpbadp-backup" not found `kbcli cluster describe-backup --names backup-ns-itqls-apemysql-zpbadp-20250911182429 --namespace ns-itqls ` Name: backup-ns-itqls-apemysql-zpbadp-20250911182429 Cluster: apemysql-zpbadp Namespace: ns-itqls Spec: Method: xtrabackup Policy Name: apemysql-zpbadp-mysql-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-backup-ns-itqls-apemysql-zpbadp-20250911182429-8be6 TargetPodName: apemysql-zpbadp-mysql-0 Phase: Completed Start Time: Sep 11,2025 18:24 UTC+0800 Completion Time: Sep 11,2025 18:24 UTC+0800 Status: Phase: Completed Total Size: 9494338 ActionSet Name: apecloud-mysql-xtrabackup Repository: backuprepo-kbcli-test Duration: 11s Start Time: Sep 11,2025 18:24 UTC+0800 Completion Time: Sep 11,2025 18:24 UTC+0800 Path: /ns-itqls/apemysql-zpbadp-3c2b36d3-b820-4f3f-b623-677fe4d89bce/mysql/backup-ns-itqls-apemysql-zpbadp-20250911182429 Time Range Start: Sep 11,2025 18:24 UTC+0800 Time Range End: Sep 11,2025 18:24 UTC+0800 Warning Events: `kbcli cluster restore apemysql-zpbadp-backup --backup backup-ns-itqls-apemysql-zpbadp-20250911182429 --namespace ns-itqls ` Cluster apemysql-zpbadp-backup created check cluster status `kbcli cluster list apemysql-zpbadp-backup --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp-backup ns-itqls apecloud-mysql WipeOut Creating Sep 11,2025 18:24 UTC+0800 clusterdefinition.kubeblocks.io/name=apecloud-mysql cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp-backup --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-backup-mysql-0 ns-itqls apemysql-zpbadp-backup mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 18:25 UTC+0800 apemysql-zpbadp-backup-mysql-1 ns-itqls apemysql-zpbadp-backup mysql Running leader 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 18:26 UTC+0800 apemysql-zpbadp-backup-mysql-2 ns-itqls apemysql-zpbadp-backup mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 18:26 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-backup-mysql-1;follower: apemysql-zpbadp-backup-mysql-0 apemysql-zpbadp-backup-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp-backup` set secret: apemysql-zpbadp-backup-mysql-account-root `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-backup-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-backup-mysql-1 --namespace ns-itqls -- bash` check cluster connect done `kbcli cluster describe-backup --names backup-ns-itqls-apemysql-zpbadp-20250911182429 --namespace ns-itqls ` Name: backup-ns-itqls-apemysql-zpbadp-20250911182429 Cluster: apemysql-zpbadp Namespace: ns-itqls Spec: Method: xtrabackup Policy Name: apemysql-zpbadp-mysql-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-backup-ns-itqls-apemysql-zpbadp-20250911182429-8be6 TargetPodName: apemysql-zpbadp-mysql-0 Phase: Completed Start Time: Sep 11,2025 18:24 UTC+0800 Completion Time: Sep 11,2025 18:24 UTC+0800 Status: Phase: Completed Total Size: 9494338 ActionSet Name: apecloud-mysql-xtrabackup Repository: backuprepo-kbcli-test Duration: 11s Start Time: Sep 11,2025 18:24 UTC+0800 Completion Time: Sep 11,2025 18:24 UTC+0800 Path: /ns-itqls/apemysql-zpbadp-3c2b36d3-b820-4f3f-b623-677fe4d89bce/mysql/backup-ns-itqls-apemysql-zpbadp-20250911182429 Time Range Start: Sep 11,2025 18:24 UTC+0800 Time Range End: Sep 11,2025 18:24 UTC+0800 Warning Events: cluster connect `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp-backup` set secret: apemysql-zpbadp-backup-mysql-account-root `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. SERVER_ID IP_PORT ROLE CONNECTED LOG_DELAY_NUM APPLY_DELAY_NUM 1 apemysql-zpbadp-backup-mysql-0.apemysql-zpbadp-backup-mysql-headless:13306 Follower YES 0 0 2 apemysql-zpbadp-backup-mysql-1.apemysql-zpbadp-backup-mysql-headless:13306 Leader YES 0 0 3 apemysql-zpbadp-backup-mysql-2.apemysql-zpbadp-backup-mysql-headless:13306 Follower YES 0 0 `echo "echo \"show variables like 'max_connections';\" | mysql -P3306 -hapemysql-zpbadp-backup-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-backup-mysql-1 --namespace ns-itqls -- bash ` Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Variable_name Value max_connections 100 connect cluster Success set max_connections to 100 delete cluster apemysql-zpbadp-backup `kbcli cluster delete apemysql-zpbadp-backup --auto-approve --namespace ns-itqls ` Cluster apemysql-zpbadp-backup deleted pod_info:apemysql-zpbadp-backup-mysql-0 4/4 Running 0 2m20s apemysql-zpbadp-backup-mysql-1 4/4 Running 0 81s apemysql-zpbadp-backup-mysql-2 4/4 Running 0 56s pod_info:apemysql-zpbadp-backup-mysql-0 4/4 Terminating 0 2m40s apemysql-zpbadp-backup-mysql-1 4/4 Terminating 0 101s apemysql-zpbadp-backup-mysql-2 4/4 Terminating 0 76s No resources found in ns-itqls namespace. delete cluster pod done No resources found in ns-itqls namespace. check cluster resource non-exist OK: pvc No resources found in ns-itqls namespace. delete cluster done No resources found in ns-itqls namespace. No resources found in ns-itqls namespace. No resources found in ns-itqls namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups backup-ns-itqls-apemysql-zpbadp-20250911182429 --namespace ns-itqls ` backup.dataprotection.kubeblocks.io/backup-ns-itqls-apemysql-zpbadp-20250911182429 patched `kbcli cluster delete-backup apemysql-zpbadp --name backup-ns-itqls-apemysql-zpbadp-20250911182429 --force --auto-approve --namespace ns-itqls ` Backup backup-ns-itqls-apemysql-zpbadp-20250911182429 deleted No opsrequests found in ns-itqls namespace. `kubectl get backupschedule -l app.kubernetes.io/instance=apemysql-zpbadp ` `kubectl get backupschedule apemysql-zpbadp-mysql-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched check backup status `kbcli cluster list-backups apemysql-zpbadp --namespace ns-itqls ` No backups found in ns-itqls namespace. No backups found in ns-itqls namespace. backup_status:-- No backups found in ns-itqls namespace. backup_status:-- No backups found in ns-itqls namespace. backup_status:-- No backups found in ns-itqls namespace. backup_status:-- No backups found in ns-itqls namespace. backup_status:-- No backups found in ns-itqls namespace. backup_status:-- No backups found in ns-itqls namespace. backup_status:-- No backups found in ns-itqls namespace. backup_status:-- `kubectl get backupschedule -l app.kubernetes.io/instance=apemysql-zpbadp ` `kubectl get backupschedule apemysql-zpbadp-mysql-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched (no change) backup_status:apemysql-zpbadp-xtrabackup-Running backup_status:apemysql-zpbadp-xtrabackup-Running check backup status done backup_status:apemysql-zpbadp-xtrabackup-20250911102900 ns-itqls apemysql-zpbadp xtrabackup Completed 6816595 11s Delete Sep 11,2025 18:29 UTC+0800 Sep 11,2025 18:29 UTC+0800 Sep 19,2025 18:29 UTC+0800 `kubectl get backupschedule -l app.kubernetes.io/instance=apemysql-zpbadp ` `kubectl get backupschedule apemysql-zpbadp-mysql-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/apemysql-zpbadp-mysql-backup-schedule patched (no change) cluster restore backup Error from server (NotFound): opsrequests.operations.kubeblocks.io "apemysql-zpbadp-backup" not found `kbcli cluster describe-backup --names apemysql-zpbadp-xtrabackup-20250911102900 --namespace ns-itqls ` Name: apemysql-zpbadp-xtrabackup-20250911102900 Cluster: apemysql-zpbadp Namespace: ns-itqls Spec: Method: xtrabackup Policy Name: apemysql-zpbadp-mysql-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-apemysql-zpbadp-xtrabackup-20250911102900-ec37a5b9 TargetPodName: apemysql-zpbadp-mysql-0 Phase: Completed Start Time: Sep 11,2025 18:29 UTC+0800 Completion Time: Sep 11,2025 18:29 UTC+0800 Status: Phase: Completed Total Size: 6816595 ActionSet Name: apecloud-mysql-xtrabackup Repository: backuprepo-kbcli-test Duration: 11s Expiration Time: Sep 19,2025 18:29 UTC+0800 Start Time: Sep 11,2025 18:29 UTC+0800 Completion Time: Sep 11,2025 18:29 UTC+0800 Path: /ns-itqls/apemysql-zpbadp-3c2b36d3-b820-4f3f-b623-677fe4d89bce/mysql/apemysql-zpbadp-xtrabackup-20250911102900 Time Range Start: Sep 11,2025 18:29 UTC+0800 Time Range End: Sep 11,2025 18:29 UTC+0800 Warning Events: `kbcli cluster restore apemysql-zpbadp-backup --backup apemysql-zpbadp-xtrabackup-20250911102900 --namespace ns-itqls ` Cluster apemysql-zpbadp-backup created check cluster status `kbcli cluster list apemysql-zpbadp-backup --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp-backup ns-itqls apecloud-mysql WipeOut Sep 11,2025 18:29 UTC+0800 cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp-backup --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-backup-mysql-0 ns-itqls apemysql-zpbadp-backup mysql Running leader 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 18:29 UTC+0800 apemysql-zpbadp-backup-mysql-1 ns-itqls apemysql-zpbadp-backup mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 18:30 UTC+0800 apemysql-zpbadp-backup-mysql-2 ns-itqls apemysql-zpbadp-backup mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 18:30 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-backup-mysql-0;follower: apemysql-zpbadp-backup-mysql-1 apemysql-zpbadp-backup-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp-backup` set secret: apemysql-zpbadp-backup-mysql-account-root `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-backup-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-backup-mysql-0 --namespace ns-itqls -- bash` check cluster connect done `kbcli cluster describe-backup --names apemysql-zpbadp-xtrabackup-20250911102900 --namespace ns-itqls ` Name: apemysql-zpbadp-xtrabackup-20250911102900 Cluster: apemysql-zpbadp Namespace: ns-itqls Spec: Method: xtrabackup Policy Name: apemysql-zpbadp-mysql-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-apemysql-zpbadp-xtrabackup-20250911102900-ec37a5b9 TargetPodName: apemysql-zpbadp-mysql-0 Phase: Completed Start Time: Sep 11,2025 18:29 UTC+0800 Completion Time: Sep 11,2025 18:29 UTC+0800 Status: Phase: Completed Total Size: 6816595 ActionSet Name: apecloud-mysql-xtrabackup Repository: backuprepo-kbcli-test Duration: 11s Expiration Time: Sep 19,2025 18:29 UTC+0800 Start Time: Sep 11,2025 18:29 UTC+0800 Completion Time: Sep 11,2025 18:29 UTC+0800 Path: /ns-itqls/apemysql-zpbadp-3c2b36d3-b820-4f3f-b623-677fe4d89bce/mysql/apemysql-zpbadp-xtrabackup-20250911102900 Time Range Start: Sep 11,2025 18:29 UTC+0800 Time Range End: Sep 11,2025 18:29 UTC+0800 Warning Events: cluster connect `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp-backup` set secret: apemysql-zpbadp-backup-mysql-account-root `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-backup-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. SERVER_ID IP_PORT ROLE CONNECTED LOG_DELAY_NUM APPLY_DELAY_NUM 1 apemysql-zpbadp-backup-mysql-0.apemysql-zpbadp-backup-mysql-headless:13306 Leader YES 0 0 2 apemysql-zpbadp-backup-mysql-1.apemysql-zpbadp-backup-mysql-headless:13306 Follower YES 0 1 3 apemysql-zpbadp-backup-mysql-2.apemysql-zpbadp-backup-mysql-headless:13306 Follower YES 0 1 `echo "echo \"show variables like 'max_connections';\" | mysql -P3306 -hapemysql-zpbadp-backup-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-backup-mysql-0 --namespace ns-itqls -- bash ` Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Variable_name Value max_connections 100 connect cluster Success set max_connections to 100 delete cluster apemysql-zpbadp-backup `kbcli cluster delete apemysql-zpbadp-backup --auto-approve --namespace ns-itqls ` Cluster apemysql-zpbadp-backup deleted pod_info:apemysql-zpbadp-backup-mysql-0 4/4 Running 0 97s apemysql-zpbadp-backup-mysql-1 4/4 Running 0 79s apemysql-zpbadp-backup-mysql-2 4/4 Running 0 51s pod_info:apemysql-zpbadp-backup-mysql-0 4/4 Terminating 0 117s apemysql-zpbadp-backup-mysql-1 4/4 Terminating 0 99s apemysql-zpbadp-backup-mysql-2 4/4 Terminating 0 71s No resources found in ns-itqls namespace. delete cluster pod done No resources found in ns-itqls namespace. check cluster resource non-exist OK: pvc No resources found in ns-itqls namespace. delete cluster done No resources found in ns-itqls namespace. No resources found in ns-itqls namespace. No resources found in ns-itqls namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups apemysql-zpbadp-xtrabackup-20250911102900 --namespace ns-itqls ` backup.dataprotection.kubeblocks.io/apemysql-zpbadp-xtrabackup-20250911102900 patched `kbcli cluster delete-backup apemysql-zpbadp --name apemysql-zpbadp-xtrabackup-20250911102900 --force --auto-approve --namespace ns-itqls ` Backup apemysql-zpbadp-xtrabackup-20250911102900 deleted No opsrequests found in ns-itqls namespace. cluster list-logs `kbcli cluster list-logs apemysql-zpbadp --namespace ns-itqls ` No log files found. Error from server (NotFound): pods "apemysql-zpbadp-mysql-1" not found cluster logs `kbcli cluster logs apemysql-zpbadp --tail 30 --namespace ns-itqls ` Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) 2025-09-11T10:17:36Z INFO event send event: map[member:apemysql-zpbadp-mysql-1 message:Failover happens, i am the healthiest member, attempt to take the leader operation:healthyCheck] 2025-09-11T10:17:36Z INFO HA Take the leader success! 2025-09-11T10:18:33Z INFO HA This member is Cluster's leader 2025-09-11T10:18:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:19:33Z INFO HA This member is Cluster's leader 2025-09-11T10:19:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:20:33Z INFO HA This member is Cluster's leader 2025-09-11T10:20:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:21:33Z INFO HA This member is Cluster's leader 2025-09-11T10:21:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:22:33Z INFO HA This member is Cluster's leader 2025-09-11T10:22:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:23:33Z INFO HA This member is Cluster's leader 2025-09-11T10:23:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:24:33Z INFO HA This member is Cluster's leader 2025-09-11T10:24:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:25:33Z INFO HA This member is Cluster's leader 2025-09-11T10:25:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:26:33Z INFO HA This member is Cluster's leader 2025-09-11T10:26:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:27:33Z INFO HA This member is Cluster's leader 2025-09-11T10:27:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:28:33Z INFO HA This member is Cluster's leader 2025-09-11T10:28:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:29:33Z INFO HA This member is Cluster's leader 2025-09-11T10:29:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:30:33Z INFO HA This member is Cluster's leader 2025-09-11T10:30:33Z DEBUG HA Refresh leader ttl 2025-09-11T10:31:33Z INFO HA This member is Cluster's leader 2025-09-11T10:31:33Z DEBUG HA Refresh leader ttl cluster logs slow `kbcli cluster logs apemysql-zpbadp --tail 30 --file-type=slow --namespace ns-itqls ` /usr/sbin/mysqld, Version: 8.0.30 (WeSQL Server - GPL, Release 5, Revision 94d1caf). started with: Tcp port: 3306 Unix socket: /var/run/mysqld/mysqld.sock Time Id Command Argument /usr/sbin/mysqld, Version: 8.0.30 (WeSQL Server - GPL, Release 5, Revision 94d1caf). started with: Tcp port: 3306 Unix socket: /var/run/mysqld/mysqld.sock Time Id Command Argument /usr/sbin/mysqld, Version: 8.0.30 (WeSQL Server - GPL, Release 5, Revision 94d1caf). started with: Tcp port: 3306 Unix socket: /var/run/mysqld/mysqld.sock Time Id Command Argument /usr/sbin/mysqld, Version: 8.0.30 (WeSQL Server - GPL, Release 5, Revision 94d1caf). started with: Tcp port: 3306 Unix socket: /var/run/mysqld/mysqld.sock Time Id Command Argument /usr/sbin/mysqld, Version: 8.0.30 (WeSQL Server - GPL, Release 5, Revision 94d1caf). started with: Tcp port: 3306 Unix socket: /var/run/mysqld/mysqld.sock Time Id Command Argument cluster logs error `kbcli cluster logs apemysql-zpbadp --tail 30 --file-type=error --namespace ns-itqls ` 2025-09-11T10:31:46.006422Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6510228 6582 2025-09-11T10:31:47.006582Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6510603 6583 2025-09-11T10:31:48.006741Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6510978 6584 2025-09-11T10:31:49.006917Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6511353 6585 2025-09-11T10:31:50.007182Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6511728 6586 2025-09-11T10:31:51.007436Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6512103 6587 2025-09-11T10:31:52.009329Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6512478 6588 2025-09-11T10:31:53.009512Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6512853 6589 2025-09-11T10:31:54.010304Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6513228 6590 2025-09-11T10:31:55.010477Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6513603 6591 2025-09-11T10:31:56.010658Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6513978 6592 2025-09-11T10:31:57.010929Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6514353 6593 2025-09-11T10:31:58.011149Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6514728 6594 2025-09-11T10:31:59.011310Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6515103 6595 2025-09-11T10:32:00.011468Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6515478 6596 2025-09-11T10:32:01.011703Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6515853 6597 2025-09-11T10:32:02.011867Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6516228 6598 2025-09-11T10:32:03.012019Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6516603 6599 2025-09-11T10:32:04.012185Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6516978 6600 2025-09-11T10:32:05.012344Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6517353 6601 2025-09-11T10:32:06.012566Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6517728 6602 2025-09-11T10:32:07.012722Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6518103 6603 2025-09-11T10:32:08.012879Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6518478 6604 2025-09-11T10:32:09.013118Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6518853 6605 2025-09-11T10:32:10.013317Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6519228 6606 2025-09-11T10:32:11.013535Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6519603 6607 2025-09-11T10:32:12.013712Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6519978 6608 2025-09-11T10:32:13.013880Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6520353 6609 2025-09-11T10:32:14.014046Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6520728 6610 2025-09-11T10:32:16.014269Z 0 [Note] [MY-000000] [Server] Report binlog commit position: /data/mysql/binlog/mysql-bin.000001 6521103 6611 LB_TYPE is set to: internet cluster expose check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster expose apemysql-zpbadp --auto-approve --force=true --type internet --enable false --components mysql --role-selector leader --namespace ns-itqls ` OpsRequest apemysql-zpbadp-expose-zdmnz created successfully, you can view the progress: kbcli cluster describe-ops apemysql-zpbadp-expose-zdmnz -n ns-itqls check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-expose-zdmnz ns-itqls Expose apemysql-zpbadp Creating -/- Sep 11,2025 18:32 UTC+0800 check cluster status `kbcli cluster list apemysql-zpbadp --show-labels --namespace ns-itqls ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS apemysql-zpbadp ns-itqls apecloud-mysql WipeOut Running Sep 11,2025 17:21 UTC+0800 app.kubernetes.io/instance=apemysql-zpbadp,clusterdefinition.kubeblocks.io/name=apecloud-mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances apemysql-zpbadp --namespace ns-itqls ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME apemysql-zpbadp-mysql-0 ns-itqls apemysql-zpbadp mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000000/10.224.0.6 Sep 11,2025 18:14 UTC+0800 apemysql-zpbadp-mysql-1 ns-itqls apemysql-zpbadp mysql Running leader 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000001/10.224.0.7 Sep 11,2025 18:13 UTC+0800 apemysql-zpbadp-mysql-2 ns-itqls apemysql-zpbadp mysql Running follower 0 200m / 200m 644245094400m / 644245094400m data:2Gi aks-cicdamdpool-40497330-vmss000002/10.224.0.5 Sep 11,2025 18:14 UTC+0800 check pod status done check cluster role check cluster role done leader: apemysql-zpbadp-mysql-1;follower: apemysql-zpbadp-mysql-0 apemysql-zpbadp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash` check cluster connect done check ops status `kbcli cluster list-ops apemysql-zpbadp --status all --namespace ns-itqls ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME apemysql-zpbadp-expose-zdmnz ns-itqls Expose apemysql-zpbadp mysql Running 0/1 Sep 11,2025 18:32 UTC+0800 ops_status:apemysql-zpbadp-expose-zdmnz ns-itqls Expose apemysql-zpbadp mysql Running 0/1 Sep 11,2025 18:32 UTC+0800 ops_status:apemysql-zpbadp-expose-zdmnz ns-itqls Expose apemysql-zpbadp mysql Running 0/1 Sep 11,2025 18:32 UTC+0800 ops_status:apemysql-zpbadp-expose-zdmnz ns-itqls Expose apemysql-zpbadp mysql Running 0/1 Sep 11,2025 18:32 UTC+0800 check ops status done ops_status:apemysql-zpbadp-expose-zdmnz ns-itqls Expose apemysql-zpbadp mysql Succeed 1/1 Sep 11,2025 18:32 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations apemysql-zpbadp-expose-zdmnz --namespace ns-itqls ` opsrequest.operations.kubeblocks.io/apemysql-zpbadp-expose-zdmnz patched `kbcli cluster delete-ops --name apemysql-zpbadp-expose-zdmnz --force --auto-approve --namespace ns-itqls ` OpsRequest apemysql-zpbadp-expose-zdmnz deleted `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t'" | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, vtablet, kbagent, config-manager, init-data (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value byxag check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hapemysql-zpbadp-mysql.ns-itqls.svc.cluster.local -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-1 --namespace ns-itqls -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=apemysql-zpbadp` set secret: apemysql-zpbadp-mysql-account-root `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets apemysql-zpbadp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:L940842zuWMRS68t;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'L940842zuWMRS68t' " | kubectl exec -it apemysql-zpbadp-mysql-0 --namespace ns-itqls -- bash ` check readonly db_client batch data Success delete cluster apemysql-zpbadp `kbcli cluster delete apemysql-zpbadp --auto-approve --namespace ns-itqls ` Cluster apemysql-zpbadp deleted pod_info:apemysql-zpbadp-mysql-0 4/4 Running 0 18m apemysql-zpbadp-mysql-1 4/4 Running 0 19m apemysql-zpbadp-mysql-2 4/4 Running 0 17m pod_info:apemysql-zpbadp-mysql-0 4/4 Terminating 0 19m apemysql-zpbadp-mysql-1 4/4 Terminating 0 20m apemysql-zpbadp-mysql-2 4/4 Terminating 0 18m No resources found in ns-itqls namespace. delete cluster pod done No resources found in ns-itqls namespace. check cluster resource non-exist OK: pvc No resources found in ns-itqls namespace. delete cluster done No resources found in ns-itqls namespace. No resources found in ns-itqls namespace. No resources found in ns-itqls namespace. ApeCloud Mysql Test Suite All Done! Test Engine: apecloud-mysql Test Type: 1 --------------------------------------ApeCloud Mysql (Topology = apecloud-mysql Replicas 3) Test Result-------------------------------------- [PASSED]|[Create]|[ComponentDefinition=apecloud-mysql-1.0.1;ComponentVersion=apecloud-mysql;ServiceVersion=8.0.30;]|[Description=Create a cluster with the specified component definition apecloud-mysql-1.0.1 and component version apecloud-mysql and service version 8.0.30] [PASSED]|[Connect]|[ComponentName=mysql]|[Description=Connect to the cluster] [PASSED]|[AddData]|[Values=byxag]|[Description=Add data to the cluster] [PASSED]|[CheckAddDataReadonly]|[Values=byxag;Role=Readonly]|[Description=Add data to the cluster readonly] [PASSED]|[Expose]|[Enable=true;TYPE=internet;ComponentName=mysql]|[Description=Expose Enable the internet service with mysql component] [PASSED]|[Reconfiguring]|[ComponentName=mysql;max_connections=1000,general_log=OFF]|[Description=Reconfiguring the cluster specify component mysql set max_connections=1000,general_log=OFF] [PASSED]|[Reconfiguring]|[ComponentName=mysql;max_connections=2000,general_log=ON,innodb_sort_buffer_size=2097152]|[Description=Reconfiguring the cluster specify component mysql set max_connections=2000,general_log=ON,innodb_sort_buffer_size=2097152] [PASSED]|[Failover]|[HA=Pod Kill;ComponentName=mysql]|[Description=Simulates conditions where pods experience kill for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to kill.] [PASSED]|[Reconfiguring]|[ComponentName=mysql;log_error_verbosity=3]|[Description=Reconfiguring the cluster specify component mysql set log_error_verbosity=3] [PASSED]|[SwitchOver]|[ComponentName=mysql]|[Description=SwitchOver the cluster specify component mysql] [PASSED]|[Bench]|[ComponentName=mysql]|[Description=Bench the cluster service with mysql component] [PASSED]|[Bench]|[HostType=LB;ComponentName=mysql]|[Description=Bench the cluster LB service with mysql component] [PASSED]|[Failover]|[HA=Network Corrupt;Durations=2m;ComponentName=mysql]|[Description=Simulates network corrupt fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to corrupt network.] [PASSED]|[VolumeExpansion]|[ComponentName=mysql]|[Description=VolumeExpansion the cluster specify component mysql] [PASSED]|[No-Failover]|[HA=DNS Random;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience random IP addresses being returned by the DNS service for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to the DNS service returning random IP addresses.] [PASSED]|[Stop]|[-]|[Description=Stop the cluster] [PASSED]|[Start]|[-]|[Description=Start the cluster] [PASSED]|[No-Failover]|[HA=Connection Stress;ComponentName=mysql]|[Description=Simulates conditions where pods experience connection stress either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Connection load.] [PASSED]|[Update]|[Monitor=true]|[Description=Update the cluster Monitor enable] [PASSED]|[Failover]|[HA=Network Loss;Durations=2m;ComponentName=mysql]|[Description=Simulates network loss fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to loss network.] [PASSED]|[Failover]|[HA=Pod Failure;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience failure for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to failure.] [PASSED]|[No-Failover]|[HA=DNS Error;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience DNS service errors for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to DNS service errors.] [PASSED]|[Failover]|[HA=Full CPU;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience CPU full either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high CPU load.] [PASSED]|[No-Failover]|[HA=Network Partition;Durations=2m;ComponentName=mysql]|[Description=Simulates network partition fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to partition network.] [PASSED]|[Failover]|[HA=OOM;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience OOM either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Memory load.] [PASSED]|[Failover]|[HA=Network Delay;Durations=2m;ComponentName=mysql]|[Description=Simulates network delay fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to delay network.] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [PASSED]|[No-Failover]|[HA=Network Duplicate;Durations=2m;ComponentName=mysql]|[Description=Simulates network duplicate fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to duplicate network.] [PASSED]|[Failover]|[HA=Network Bandwidth;Durations=2m;ComponentName=mysql]|[Description=Simulates network bandwidth fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to bandwidth network.] [PASSED]|[Failover]|[HA=Kill 1;ComponentName=mysql]|[Description=Simulates conditions where process 1 killed either due to expected/undesired processes thereby testing the application's resilience to unavailability of some replicas due to abnormal termination signals.] [PASSED]|[VerticalScaling]|[ComponentName=mysql]|[Description=VerticalScaling the cluster specify component mysql] [PASSED]|[No-Failover]|[HA=Time Offset;Durations=2m;ComponentName=mysql]|[Description=Simulates a time offset scenario thereby testing the application's resilience to potential slowness/unavailability of some replicas due to time offset.] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Backup]|[BackupMethod=volume-snapshot]|[Description=The cluster volume-snapshot Backup] [PASSED]|[Restore]|[BackupMethod=volume-snapshot]|[Description=The cluster volume-snapshot Restore] [PASSED]|[Connect]|[ComponentName=mysql]|[Description=Connect to the cluster] [PASSED]|[Delete Restore Cluster]|[BackupMethod=volume-snapshot]|[Description=Delete the volume-snapshot restore cluster] [PASSED]|[Backup]|[BackupMethod=xtrabackup]|[Description=The cluster xtrabackup Backup] [PASSED]|[Restore]|[BackupMethod=xtrabackup]|[Description=The cluster xtrabackup Restore] [PASSED]|[Connect]|[ComponentName=mysql]|[Description=Connect to the cluster] [PASSED]|[Delete Restore Cluster]|[BackupMethod=xtrabackup]|[Description=Delete the xtrabackup restore cluster] [PASSED]|[Backup]|[Schedule=true;BackupMethod=xtrabackup]|[Description=The cluster Schedule xtrabackup Backup] [PASSED]|[Restore]|[Schedule=true;BackupMethod=xtrabackup]|[Description=The cluster Schedule xtrabackup Restore] [PASSED]|[Connect]|[ComponentName=mysql]|[Description=Connect to the cluster] [PASSED]|[Delete Restore Cluster]|[Schedule=true;BackupMethod=xtrabackup]|[Description=Delete the Schedule xtrabackup restore cluster] [PASSED]|[Expose]|[Disable=true;TYPE=internet;ComponentName=mysql]|[Description=Expose Disable the internet service with mysql component] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]