source commons files source engines files source kubeblocks files `kubectl get namespace | grep ns-xosbc ` `kubectl create namespace ns-xosbc` namespace/ns-xosbc created create namespace ns-xosbc done download kbcli `gh release list --repo apecloud/kbcli --limit 100 | (grep "0.9" || true)` `curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v0.9.4-beta.1` Your system is linux_amd64 Installing kbcli ... Downloading ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 32.1M 100 32.1M 0 0 65.2M 0 --:--:-- --:--:-- --:--:-- 65.2M kbcli installed successfully. Kubernetes: v1.32.5-eks-5d4a308 KubeBlocks: 0.9.4 kbcli: 0.9.4-beta.1 WARNING: version difference between kbcli (0.9.4-beta.1) and kubeblocks (0.9.4) Make sure your docker service is running and begin your journey with kbcli: kbcli playground init For more information on how to get started, please visit: https://kubeblocks.io download kbcli v0.9.4-beta.1 done Kubernetes: v1.32.5-eks-5d4a308 KubeBlocks: 0.9.4 kbcli: 0.9.4-beta.1 WARNING: version difference between kbcli (0.9.4-beta.1) and kubeblocks (0.9.4) Kubernetes Env: v1.32.5-eks-5d4a308 POD_RESOURCES: No resources found found default storage class: gp3 kubeblocks version is:0.9.4 skip upgrade kubeblocks Error: no repositories to show helm repo add chaos-mesh https://charts.chaos-mesh.org "chaos-mesh" has been added to your repositories add helm chart repo chaos-mesh success chaos mesh already installed check cluster definition set component name:mysql set component version set component version:mysql set service versions:5.7.44,8.0.30,8.0.31,8.0.32,8.0.33,8.0.34,8.0.35,8.0.36,8.0.37,8.0.38,8.0.39,8.4.0,8.4.1,8.4.2 set service versions sorted:5.7.44,8.0.30,8.0.31,8.0.32,8.0.33,8.0.34,8.0.35,8.0.36,8.0.37,8.0.38,8.0.39,8.4.0,8.4.1,8.4.2 unsupported component definition set replicas first:2,5.7.44|2,8.0.30|2,8.0.31|2,8.0.32|2,8.0.33|2,8.0.34|2,8.0.35|2,8.0.36|2,8.0.37|2,8.0.38|2,8.0.39|2,8.4.0|2,8.4.1|2,8.4.2 set replicas third:2,8.0.38 set replicas fourth:2,8.0.30 set minimum cmpv service version set minimum cmpv service version replicas:2,8.0.30 REPORT_COUNT:1 CLUSTER_TOPOLOGY:replication Not found topology in cluster definition mysql LIMIT_CPU:0.1 LIMIT_MEMORY:0.5 storage size: 10 No resources found in ns-xosbc namespace. termination_policy:Delete create 2 replica Delete mysql cluster check cluster version check cluster definition check component definition set component definition by component version check cmpd by labels set component definition1: mysql-8.0 by component version:mysql apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: name: mysql-qzgper namespace: ns-xosbc spec: terminationPolicy: Delete componentSpecs: - name: mysql componentDef: mysql-8.0 serviceVersion: 8.0.30 replicas: 2 resources: requests: cpu: 100m memory: 0.5Gi limits: cpu: 100m memory: 0.5Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi `kubectl apply -f test_create_mysql-qzgper.yaml` cluster.apps.kubeblocks.io/mysql-qzgper created apply test_create_mysql-qzgper.yaml Success `rm -rf test_create_mysql-qzgper.yaml` check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Jun 19,2025 18:16 UTC+0800 cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:16 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check pod mysql-qzgper-mysql-1 container_name mysql exist password czo40138CR check pod mysql-qzgper-mysql-1 container_name lorry exist password czo40138CR check pod mysql-qzgper-mysql-1 container_name config-manager exist password czo40138CR No container logs contain secret password. describe cluster `kbcli cluster describe mysql-qzgper --namespace ns-xosbc ` Name: mysql-qzgper Created Time: Jun 19,2025 18:16 UTC+0800 NAMESPACE CLUSTER-DEFINITION VERSION STATUS TERMINATION-POLICY ns-xosbc Running Delete Endpoints: COMPONENT MODE INTERNAL EXTERNAL mysql ReadWrite mysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local:3306 mysql-qzgper-mysql-mysql-1.ns-xosbc.svc.cluster.local:3306 mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local:3306 Topology: COMPONENT INSTANCE ROLE STATUS AZ NODE CREATED-TIME mysql mysql-qzgper-mysql-0 secondary Running us-west-2a ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:16 UTC+0800 mysql mysql-qzgper-mysql-1 primary Running us-west-2a ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 Resources Allocation: COMPONENT DEDICATED CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS mysql false 100m / 100m 512Mi / 512Mi data:10Gi kb-default-sc Images: COMPONENT TYPE IMAGE mysql docker.io/apecloud/mysql:8.0.30 Show cluster events: kbcli cluster list-events -n ns-xosbc mysql-qzgper `kbcli cluster label mysql-qzgper app.kubernetes.io/instance- --namespace ns-xosbc ` label "app.kubernetes.io/instance" not found. `kbcli cluster label mysql-qzgper app.kubernetes.io/instance=mysql-qzgper --namespace ns-xosbc ` `kbcli cluster label mysql-qzgper --list --namespace ns-xosbc ` NAME NAMESPACE LABELS mysql-qzgper ns-xosbc app.kubernetes.io/instance=mysql-qzgper label cluster app.kubernetes.io/instance=mysql-qzgper Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=mysql-qzgper --namespace ns-xosbc ` `kbcli cluster label mysql-qzgper --list --namespace ns-xosbc ` NAME NAMESPACE LABELS mysql-qzgper ns-xosbc app.kubernetes.io/instance=mysql-qzgper case.name=kbcli.test1 label cluster case.name=kbcli.test1 Success `kbcli cluster label mysql-qzgper case.name=kbcli.test2 --overwrite --namespace ns-xosbc ` `kbcli cluster label mysql-qzgper --list --namespace ns-xosbc ` NAME NAMESPACE LABELS mysql-qzgper ns-xosbc app.kubernetes.io/instance=mysql-qzgper case.name=kbcli.test2 label cluster case.name=kbcli.test2 Success `kbcli cluster label mysql-qzgper case.name- --namespace ns-xosbc ` `kbcli cluster label mysql-qzgper --list --namespace ns-xosbc ` NAME NAMESPACE LABELS mysql-qzgper ns-xosbc app.kubernetes.io/instance=mysql-qzgper delete cluster label case.name Success cluster connect Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. *************************** 1. row *************************** Replica_IO_State: Waiting for source to send event Source_Host: mysql-qzgper-mysql-1.mysql-qzgper-mysql-headless Source_User: root Source_Port: 3306 Connect_Retry: 60 Source_Log_File: mysql-qzgper-mysql-1-bin.000003 Read_Source_Log_Pos: 22301 Relay_Log_File: relay-bin.000002 Relay_Log_Pos: 22146 Relay_Source_Log_File: mysql-qzgper-mysql-1-bin.000003 Replica_IO_Running: Yes Replica_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Source_Log_Pos: 21900 Relay_Log_Space: 22751 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Source_SSL_Allowed: Yes Source_SSL_CA_File: Source_SSL_CA_Path: Source_SSL_Cert: Source_SSL_Cipher: Source_SSL_Key: Seconds_Behind_Source: 0 Source_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Source_Server_Id: 2 Source_UUID: ba378b06-4cf6-11f0-8d35-3a0c1a099b0d Source_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Replica_SQL_Running_State: Replica has read all relay log; waiting for more updates Source_Retry_Count: 86400 Source_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Source_SSL_Crl: Source_SSL_Crlpath: Retrieved_Gtid_Set: ba378b06-4cf6-11f0-8d35-3a0c1a099b0d:1-56 Executed_Gtid_Set: ba378b06-4cf6-11f0-8d35-3a0c1a099b0d:1-55 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Source_TLS_Version: Source_public_key_path: Get_Source_public_key: 0 Network_Namespace: `echo 'show variables like "max_connections";' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Connect to instance mysql-qzgper-mysql-1: out of mysql-qzgper-mysql-1(primary), mysql-qzgper-mysql-0(secondary) Variable_name Value max_connections 83 connect cluster Success set max_connections to 83 insert batch data by db client Error from server (NotFound): pods "test-db-client-executionloop-mysql-qzgper" not found DB_CLIENT_BATCH_DATA_COUNT: `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-mysql-qzgper --namespace ns-xosbc ` Error from server (NotFound): pods "test-db-client-executionloop-mysql-qzgper" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-executionloop-mysql-qzgper" not found `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql No resources found in ns-xosbc namespace. apiVersion: v1 kind: Pod metadata: name: test-db-client-executionloop-mysql-qzgper namespace: ns-xosbc spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local" - "--user" - "root" - "--password" - "czo40138CR" - "--port" - "3306" - "--dbtype" - "mysql" - "--test" - "executionloop" - "--duration" - "60" - "--interval" - "1" restartPolicy: Never `kubectl apply -f test-db-client-executionloop-mysql-qzgper.yaml` pod/test-db-client-executionloop-mysql-qzgper created apply test-db-client-executionloop-mysql-qzgper.yaml Success `rm -rf test-db-client-executionloop-mysql-qzgper.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 0/1 ContainerCreating 0 6s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 0/1 ContainerCreating 0 10s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 1/1 Running 0 16s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 1/1 Running 0 22s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 1/1 Running 0 28s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 1/1 Running 0 33s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 1/1 Running 0 39s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 1/1 Running 0 45s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 1/1 Running 0 51s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 1/1 Running 0 56s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 1/1 Running 0 62s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 1/1 Running 0 68s check pod test-db-client-executionloop-mysql-qzgper status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-qzgper 0/1 Completed 0 74s check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:16 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done --host mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local --user root --password czo40138CR --port 3306 --dbtype mysql --test executionloop --duration 60 --interval 1 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider] 10:20:34.727 [main] DEBUG com.clickhouse.jdbc.ClickHouseDriver -- ClickHouse Driver 0.0.0.0(JDBC: 0.0.0.0) registered Execution loop start: create databases executions_loop CREATE DATABASE IF NOT EXISTS executions_loop; drop table executions_loop_table DROP TABLE IF EXISTS executions_loop.executions_loop_table; create table executions_loop_table CREATE TABLE IF NOT EXISTS executions_loop.executions_loop_table (id INT PRIMARY KEY AUTO_INCREMENT, value VARCHAR(255), tinyint_col TINYINT, smallint_col SMALLINT, mediumint_col MEDIUMINT, int_col INT, bigint_col BIGINT, float_col FLOAT, double_col DOUBLE, decimal_col DECIMAL(10, 2), date_col DATE, time_col TIME, datetime_col DATETIME, timestamp_col TIMESTAMP, year_col YEAR, char_col CHAR(10), text_col TEXT, blob_col BLOB, binary_col BINARY(10), varbinary_col VARBINARY(255), enum_col ENUM('Option1', 'Option2', 'Option3'), set_col SET('Value1', 'Value2', 'Value3') ); Execution loop start:INSERT INTO executions_loop.executions_loop_table (value, tinyint_col, smallint_col, mediumint_col, int_col, bigint_col, float_col, double_col, decimal_col, date_col, time_col, datetime_col, timestamp_col, year_col, char_col, text_col, blob_col, binary_col, varbinary_col, enum_col, set_col) VALUES ('executions_loop_test_1', 15, 27253, 7976908, 1374125124, -3476128781856611868, 0.95300174, 0.5089258839054291, 47.285238357441195, '2025-06-19', '10:20:35', '2025-06-19 10:20:35.446', CURRENT_TIMESTAMP, 2153, 'slYnE323Xp', 'lFJB2LuR1V8cXBUjwavqCloCxi3TVqL2QAzNMd7VUMhUkbl7sb1liDQq4n5vfB8oGAqi5epAvl4JBvQcy55r0ExmCsQoL1DqihDzDUvv4FllWTRVbPkQMf65dBfDAqyQG0j0mbIit5yB2NQ9lYJgUMjMmJX6MQYLwL0V19UrwsNn59LOLMB8LmSPKlFldTnQxzptOiwZE5OTdTawGRmcrZESGgBVJHIAei9GptRxUra4t96sKKxCEdTKvyqetSy', UNHEX(REPLACE('a5d1504d0c0513bcfbe7', ' ', '')), UNHEX(REPLACE('0288e340161471864276', ' ', '')), UNHEX(REPLACE('9828e842a41e040fba54e9aca3f62aa7120fac45601efb80f14c39dc68c48cd438cbdfc7faafbe91cd3bfa3cdfc99e8e1a839a90128f0ed03de328c2c7bdfc5fcf8eb8d39894fb506eb32f6edd7464cc712c397e7df438cf6dc8dd9fb638f1086cf0f4daae22572c109e1dabcf37b5b5f875141e3d9393a6169e08b1e262f1b0e3132366e1c9e7b88dec22657a4b5def939911eda06cb52673165df4c9d7f5914b5839a6ca4d1480a0862f3cb9dfe281adfcaf1029a1a202e2e4fdb64007714b7ffc400b75d0061c69c32e25f8691ef3b19c69710da355b2653e46daeeeb8d189ed300f3e4ac865aed711869b9431e04928b0433b9384ca922ea120cce111f', ' ', '')), 'Option1', 'Value1' ); [ 1s ] executions total: 24 successful: 24 failed: 0 disconnect: 0 [ 2s ] executions total: 47 successful: 47 failed: 0 disconnect: 0 [ 3s ] executions total: 72 successful: 72 failed: 0 disconnect: 0 [ 4s ] executions total: 97 successful: 97 failed: 0 disconnect: 0 [ 5s ] executions total: 126 successful: 126 failed: 0 disconnect: 0 [ 6s ] executions total: 150 successful: 150 failed: 0 disconnect: 0 [ 7s ] executions total: 177 successful: 177 failed: 0 disconnect: 0 [ 8s ] executions total: 206 successful: 206 failed: 0 disconnect: 0 [ 9s ] executions total: 232 successful: 232 failed: 0 disconnect: 0 [ 10s ] executions total: 258 successful: 258 failed: 0 disconnect: 0 [ 11s ] executions total: 282 successful: 282 failed: 0 disconnect: 0 [ 12s ] executions total: 308 successful: 308 failed: 0 disconnect: 0 [ 13s ] executions total: 333 successful: 333 failed: 0 disconnect: 0 [ 14s ] executions total: 360 successful: 360 failed: 0 disconnect: 0 [ 15s ] executions total: 385 successful: 385 failed: 0 disconnect: 0 [ 16s ] executions total: 412 successful: 412 failed: 0 disconnect: 0 [ 17s ] executions total: 441 successful: 441 failed: 0 disconnect: 0 [ 18s ] executions total: 467 successful: 467 failed: 0 disconnect: 0 [ 19s ] executions total: 500 successful: 500 failed: 0 disconnect: 0 [ 20s ] executions total: 527 successful: 527 failed: 0 disconnect: 0 [ 21s ] executions total: 552 successful: 552 failed: 0 disconnect: 0 [ 22s ] executions total: 576 successful: 576 failed: 0 disconnect: 0 [ 23s ] executions total: 605 successful: 605 failed: 0 disconnect: 0 [ 24s ] executions total: 633 successful: 633 failed: 0 disconnect: 0 [ 25s ] executions total: 659 successful: 659 failed: 0 disconnect: 0 [ 26s ] executions total: 687 successful: 687 failed: 0 disconnect: 0 [ 27s ] executions total: 714 successful: 714 failed: 0 disconnect: 0 [ 28s ] executions total: 737 successful: 737 failed: 0 disconnect: 0 [ 29s ] executions total: 761 successful: 761 failed: 0 disconnect: 0 [ 30s ] executions total: 791 successful: 791 failed: 0 disconnect: 0 [ 31s ] executions total: 820 successful: 820 failed: 0 disconnect: 0 [ 32s ] executions total: 853 successful: 853 failed: 0 disconnect: 0 [ 33s ] executions total: 880 successful: 880 failed: 0 disconnect: 0 [ 34s ] executions total: 910 successful: 910 failed: 0 disconnect: 0 [ 35s ] executions total: 941 successful: 941 failed: 0 disconnect: 0 [ 36s ] executions total: 966 successful: 966 failed: 0 disconnect: 0 [ 37s ] executions total: 999 successful: 999 failed: 0 disconnect: 0 [ 38s ] executions total: 1030 successful: 1030 failed: 0 disconnect: 0 [ 39s ] executions total: 1061 successful: 1061 failed: 0 disconnect: 0 [ 40s ] executions total: 1092 successful: 1092 failed: 0 disconnect: 0 [ 41s ] executions total: 1118 successful: 1118 failed: 0 disconnect: 0 [ 42s ] executions total: 1142 successful: 1142 failed: 0 disconnect: 0 [ 43s ] executions total: 1171 successful: 1171 failed: 0 disconnect: 0 [ 44s ] executions total: 1202 successful: 1202 failed: 0 disconnect: 0 [ 45s ] executions total: 1227 successful: 1227 failed: 0 disconnect: 0 [ 46s ] executions total: 1259 successful: 1259 failed: 0 disconnect: 0 [ 47s ] executions total: 1291 successful: 1291 failed: 0 disconnect: 0 [ 48s ] executions total: 1324 successful: 1324 failed: 0 disconnect: 0 [ 49s ] executions total: 1357 successful: 1357 failed: 0 disconnect: 0 [ 50s ] executions total: 1389 successful: 1389 failed: 0 disconnect: 0 [ 51s ] executions total: 1420 successful: 1420 failed: 0 disconnect: 0 [ 52s ] executions total: 1451 successful: 1451 failed: 0 disconnect: 0 [ 53s ] executions total: 1482 successful: 1482 failed: 0 disconnect: 0 [ 54s ] executions total: 1513 successful: 1513 failed: 0 disconnect: 0 [ 55s ] executions total: 1544 successful: 1544 failed: 0 disconnect: 0 [ 56s ] executions total: 1577 successful: 1577 failed: 0 disconnect: 0 [ 57s ] executions total: 1609 successful: 1609 failed: 0 disconnect: 0 [ 58s ] executions total: 1638 successful: 1638 failed: 0 disconnect: 0 [ 60s ] executions total: 1672 successful: 1672 failed: 0 disconnect: 0 Test Result: Total Executions: 1672 Successful Executions: 1672 Failed Executions: 0 Disconnection Counts: 0 Connection Information: Database Type: mysql Host: mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local Port: 3306 Database: Table: User: root Org: Access Mode: mysql Test Type: executionloop Query: Duration: 60 seconds Interval: 1 seconds DB_CLIENT_BATCH_DATA_COUNT: 1672 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-mysql-qzgper --namespace ns-xosbc ` pod/test-db-client-executionloop-mysql-qzgper patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-executionloop-mysql-qzgper" force deleted `echo "CREATE DATABASE IF NOT EXISTS mydb; use mydb; DROP TABLE IF EXISTS tmp_table; CREATE TABLE IF NOT EXISTS tmp_table (id INT PRIMARY KEY AUTO_INCREMENT, value VARCHAR(255)); INSERT INTO tmp_table (value) VALUES ('llsap');" | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` Connect to instance mysql-qzgper-mysql-1: out of mysql-qzgper-mysql-1(primary), mysql-qzgper-mysql-0(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. add consistent data llsap Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"CREATE DATABASE IF NOT EXISTS mydb; use mydb; DROP TABLE IF EXISTS tmp_table; CREATE TABLE IF NOT EXISTS tmp_table (id INT PRIMARY KEY AUTO_INCREMENT, value VARCHAR(255)); INSERT INTO tmp_table (value) VALUES ('llsap');\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 1290 (HY000) at line 1: The MySQL server is running with the --super-read-only option so it cannot execute this statement command terminated with exit code 1 check add consistent data readonly Success cluster expose check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster expose mysql-qzgper --auto-approve --force=true --type internet --enable true --components mysql --namespace ns-xosbc ` OpsRequest mysql-qzgper-expose-64xzh created successfully, you can view the progress: kbcli cluster describe-ops mysql-qzgper-expose-64xzh -n ns-xosbc check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-expose-64xzh ns-xosbc Expose mysql-qzgper mysql Running 0/1 Jun 19,2025 18:22 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:16 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-expose-64xzh ns-xosbc Expose mysql-qzgper mysql Succeed 1/1 Jun 19,2025 18:22 UTC+0800 check ops status done ops_status:mysql-qzgper-expose-64xzh ns-xosbc Expose mysql-qzgper mysql Succeed 1/1 Jun 19,2025 18:22 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-expose-64xzh --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-expose-64xzh patched `kbcli cluster delete-ops --name mysql-qzgper-expose-64xzh --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-expose-64xzh deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-1: out of mysql-qzgper-mysql-1(primary), mysql-qzgper-mysql-0(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover oom check node drain check node drain success Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-qzgper --namespace ns-xosbc ` Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-qzgper" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-1 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-mysql-qzgper.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mysql-qzgper created apply test-chaos-mesh-oom-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-oom-mysql-qzgper.yaml` check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:16 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-qzgper" force deleted stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mysql-qzgper patched check failover pod name failover pod name:mysql-qzgper-mysql-0 failover oom Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover dnserror check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mysql-qzgper" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnserror-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-0 mode: all action: error duration: 2m `kubectl apply -f test-chaos-mesh-dnserror-mysql-qzgper.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnserror-mysql-qzgper created apply test-chaos-mesh-dnserror-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-dnserror-mysql-qzgper.yaml` dnserror chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:16 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org/test-chaos-mesh-dnserror-mysql-qzgper patched check failover pod name dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mysql-qzgper" force deleted failover pod name:mysql-qzgper-mysql-0 failover dnserror Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success cluster update monitor KUBEBLOCKS NAMESPACE:kb-mqtcr get kubeblocks namespace done check addon prometheus enabled check addon prometheus enabled done check addon grafana enabled check addon grafana enabled done check addon apecloud-otel-collector enabled check addon apecloud-otel-collector enabled done kubectl patch cluster mysql-qzgper --namespace ns-xosbc --type json -p '[***"op": "replace", "path": "/spec/componentSpecs/0/disableExporter", "value": false***]' cluster.apps.kubeblocks.io/mysql-qzgper patched check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:16 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done create curl-jq pod mysql-qzgper-kb-curl `kubectl run --namespace kb-mqtcr mysql-qzgper-kb-curl --restart='Never' --image docker.io/apecloud/curl-jq:0.1.0 --command -- sleep infinity ` pod/mysql-qzgper-kb-curl created check pod status pod_status:NAME READY STATUS RESTARTS AGE mysql-qzgper-kb-curl 0/1 ContainerCreating 0 1s check pod mysql-qzgper-kb-curl status done `kubectl wait --for=condition=Ready pod mysql-qzgper-kb-curl --namespace kb-mqtcr` pod/mysql-qzgper-kb-curl condition met pod_status:NAME READY STATUS RESTARTS AGE mysql-qzgper-kb-curl 1/1 Running 0 6s kubectl exec -it mysql-qzgper-kb-curl --namespace kb-mqtcr -- curl -G "http://kb-addon-prometheus-server:80/api/v1/query" -d "query=sum(k8s_pod_cpu_limit)" `kubectl exec -it mysql-qzgper-kb-curl --namespace kb-mqtcr -- curl -G "http://kb-addon-prometheus-server:80/api/v1/query" -d "query=sum(k8s_pod_cpu_limit)"` Unable to use a TTY - input is not a terminal or the right kind of file % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 122 100 122 0 0 22651 0 --:--:-- --:--:-- --:--:-- 24400 monitor_result:***"status":"success","data":***"resultType":"vector","result":[***"metric":***,"value":[1750328795.247,"24.200000000000017"]***]*** k8s_pod_cpu_limit: 1750328795.247 monitor cluster Success `kubectl delete pod mysql-qzgper-kb-curl --namespace kb-mqtcr --force` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "mysql-qzgper-kb-curl" force deleted check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:16 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done test failover networkduplicate check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkduplicate-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-0 mode: all action: duplicate duplicate: duplicate: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkduplicate-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkduplicate-mysql-qzgper created apply test-chaos-mesh-networkduplicate-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networkduplicate-mysql-qzgper.yaml` networkduplicate chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:16 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mysql-qzgper" not found check failover pod name failover pod name:mysql-qzgper-mysql-0 failover networkduplicate Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover fullcpu check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mysql-qzgper" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-fullcpu-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-0 mode: all stressors: cpu: workers: 100 load: 100 duration: 2m `kubectl apply -f test-chaos-mesh-fullcpu-mysql-qzgper.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-fullcpu-mysql-qzgper created apply test-chaos-mesh-fullcpu-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-fullcpu-mysql-qzgper.yaml` fullcpu chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 18:16 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mysql-qzgper" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mysql-qzgper" not found check failover pod name failover pod name:mysql-qzgper-mysql-0 failover fullcpu Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover drainnode check node drain check node drain success kubectl get pod mysql-qzgper-mysql-0 --namespace ns-xosbc -o jsonpath='***.spec.nodeName***' get node name:ip-172-31-10-206.us-west-2.compute.internal success check if multiple pods are on the same node kubectl get pod mysql-qzgper-mysql-1 --namespace ns-xosbc -o jsonpath='***.spec.nodeName***' get node name:ip-172-31-8-116.us-west-2.compute.internal success kubectl drain ip-172-31-10-206.us-west-2.compute.internal --delete-emptydir-data --ignore-daemonsets --force --grace-period 0 --timeout 60s node/ip-172-31-10-206.us-west-2.compute.internal cordoned Warning: ignoring DaemonSet-managed Pods: chaos-mesh/chaos-daemon-2clvr, kb-mqtcr/kb-addon-apecloud-otel-collector-sg5pq, kube-system/aws-node-rws5g, kube-system/ebs-csi-node-bn65b, kube-system/kube-proxy-rfmmk evicting pod ns-xosbc/mysql-qzgper-mysql-0 evicting pod chaos-mesh/chaos-controller-manager-9bb99f55c-c8z4n evicting pod ns-jcjpx/redis-ljzvzz-redis-twemproxy-0 evicting pod ns-svryh/kingbase-jzvvix-kingbase-0 pod/redis-ljzvzz-redis-twemproxy-0 evicted pod/kingbase-jzvvix-kingbase-0 evicted pod/mysql-qzgper-mysql-0 evicted pod/chaos-controller-manager-9bb99f55c-c8z4n evicted node/ip-172-31-10-206.us-west-2.compute.internal drained kubectl uncordon ip-172-31-10-206.us-west-2.compute.internal node/ip-172-31-10-206.us-west-2.compute.internal uncordoned check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-144.us-west-2.compute.internal/172.31.5.144 Jun 19,2025 18:31 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done check failover pod name failover pod name:mysql-qzgper-mysql-1 failover drainnode Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-1: out of mysql-qzgper-mysql-1(primary), mysql-qzgper-mysql-0(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover connectionstress check node drain check node drain success Error from server (NotFound): pods "test-db-client-connectionstress-mysql-qzgper" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstress-mysql-qzgper --namespace ns-xosbc ` Error from server (NotFound): pods "test-db-client-connectionstress-mysql-qzgper" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-connectionstress-mysql-qzgper" not found `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql No resources found in ns-xosbc namespace. apiVersion: v1 kind: Pod metadata: name: test-db-client-connectionstress-mysql-qzgper namespace: ns-xosbc spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local" - "--user" - "root" - "--password" - "czo40138CR" - "--port" - "3306" - "--database" - "mysql" - "--dbtype" - "mysql" - "--test" - "connectionstress" - "--connections" - "83" - "--duration" - "60" restartPolicy: Never `kubectl apply -f test-db-client-connectionstress-mysql-qzgper.yaml` pod/test-db-client-connectionstress-mysql-qzgper created apply test-db-client-connectionstress-mysql-qzgper.yaml Success `rm -rf test-db-client-connectionstress-mysql-qzgper.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mysql-qzgper 0/1 ContainerCreating 0 6s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mysql-qzgper 1/1 Running 0 11s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mysql-qzgper 1/1 Running 0 17s check pod test-db-client-connectionstress-mysql-qzgper status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mysql-qzgper 0/1 Completed 0 23s check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-144.us-west-2.compute.internal/172.31.5.144 Jun 19,2025 18:31 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done 10:34:23.168 [main] DEBUG com.yashandb.log.LoggerFactory -- Logger type is SLF4J Logger. 10:34:23.170 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:mysql://mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local:3306/mysql?useSSL=false&allowPublicKeyRetrieval=true Failed to connect to MySQL database: java.sql.SQLNonTransientConnectionException: Data source rejected establishment of connection, message from server: "Too many connections" Trying with database MySQL. 10:34:23.174 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:mysql://mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local:3306?useSSL=false&allowPublicKeyRetrieval=true java.io.IOException: Failed to connect to MySQL database: at com.apecloud.dbtester.tester.MySQLTester.connect(MySQLTester.java:63) at com.apecloud.dbtester.tester.MySQLTester.connectionStress(MySQLTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: java.sql.SQLNonTransientConnectionException: Data source rejected establishment of connection, message from server: "Too many connections" at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:829) at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:449) at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:242) at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.MySQLTester.connect(MySQLTester.java:61) ... 4 more 10:34:23.178 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:mysql://mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local:3306/mysql?useSSL=false&allowPublicKeyRetrieval=true Failed to connect to MySQL database: java.sql.SQLNonTransientConnectionException: Data source rejected establishment of connection, message from server: "Too many connections" Trying with database MySQL. 10:34:23.181 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:mysql://mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local:3306?useSSL=false&allowPublicKeyRetrieval=true java.io.IOException: Failed to connect to MySQL database: at com.apecloud.dbtester.tester.MySQLTester.connect(MySQLTester.java:63) at com.apecloud.dbtester.tester.MySQLTester.connectionStress(MySQLTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: java.sql.SQLNonTransientConnectionException: Data source rejected establishment of connection, message from server: "Too many connections" at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:829) at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:449) at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:242) at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.MySQLTester.connect(MySQLTester.java:61) ... 4 more 10:34:23.185 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:mysql://mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local:3306/mysql?useSSL=false&allowPublicKeyRetrieval=true Failed to connect to MySQL database: java.sql.SQLNonTransientConnectionException: Data source rejected establishment of connection, message from server: "Too many connections" Trying with database MySQL. 10:34:23.187 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:mysql://mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local:3306?useSSL=false&allowPublicKeyRetrieval=true java.io.IOException: Failed to connect to MySQL database: at com.apecloud.dbtester.tester.MySQLTester.connect(MySQLTester.java:63) at com.apecloud.dbtester.tester.MySQLTester.connectionStress(MySQLTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: java.sql.SQLNonTransientConnectionException: Data source rejected establishment of connection, message from server: "Too many connections" at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:829) at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:449) at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:242) at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.MySQLTester.connect(MySQLTester.java:61) ... 4 more 10:34:23.191 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:mysql://mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local:3306/mysql?useSSL=false&allowPublicKeyRetrieval=true Failed to connect to MySQL database: java.sql.SQLNonTransientConnectionException: Data source rejected establishment of connection, message from server: "Too many connections" Trying with database MySQL. 10:34:23.193 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:mysql://mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local:3306?useSSL=false&allowPublicKeyRetrieval=true java.io.IOException: Failed to connect to MySQL database: at com.apecloud.dbtester.tester.MySQLTester.connect(MySQLTester.java:63) at com.apecloud.dbtester.tester.MySQLTester.connectionStress(MySQLTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: java.sql.SQLNonTransientConnectionException: Data source rejected establishment of connection, message from server: "Too many connections" at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:829) at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:449) at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:242) at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.MySQLTester.connect(MySQLTester.java:61) ... 4 more Test Result: null Connection Information: Database Type: mysql Host: mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local Port: 3306 Database: mysql Table: User: root Org: Access Mode: mysql Test Type: connectionstress Connection Count: 83 Duration: 60 seconds `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstress-mysql-qzgper --namespace ns-xosbc ` pod/test-db-client-connectionstress-mysql-qzgper patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-connectionstress-mysql-qzgper" force deleted check failover pod name failover pod name:mysql-qzgper-mysql-1 failover connectionstress Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-1: out of mysql-qzgper-mysql-1(primary), mysql-qzgper-mysql-0(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success cluster update enable-all-logs `kbcli cluster update mysql-qzgper --enable-all-logs=true --namespace ns-xosbc ` error: name is required check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-144.us-west-2.compute.internal/172.31.5.144 Jun 19,2025 18:31 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done test failover networkdelay check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkdelay-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-1 mode: all action: delay delay: latency: 2000ms correlation: '100' jitter: 0ms direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkdelay-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkdelay-mysql-qzgper created apply test-chaos-mesh-networkdelay-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networkdelay-mysql-qzgper.yaml` networkdelay chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-144.us-west-2.compute.internal/172.31.5.144 Jun 19,2025 18:31 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mysql-qzgper" not found check failover pod name failover pod name:mysql-qzgper-mysql-1 failover networkdelay Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-1: out of mysql-qzgper-mysql-1(primary), mysql-qzgper-mysql-0(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover podfailure check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mysql-qzgper" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podfailure-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-1 mode: all action: pod-failure duration: 2m `kubectl apply -f test-chaos-mesh-podfailure-mysql-qzgper.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podfailure-mysql-qzgper created apply test-chaos-mesh-podfailure-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-podfailure-mysql-qzgper.yaml` podfailure chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-144.us-west-2.compute.internal/172.31.5.144 Jun 19,2025 18:31 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-8-116.us-west-2.compute.internal/172.31.8.116 Jun 19,2025 18:16 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mysql-qzgper" force deleted Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mysql-qzgper" not found check failover pod name failover pod name:mysql-qzgper-mysql-0 failover podfailure Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart mysql-qzgper --auto-approve --force=true --namespace ns-xosbc ` OpsRequest mysql-qzgper-restart-wk8sz created successfully, you can view the progress: kbcli cluster describe-ops mysql-qzgper-restart-wk8sz -n ns-xosbc check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-restart-wk8sz ns-xosbc Restart mysql-qzgper mysql Running 0/2 Jun 19,2025 18:40 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-restart-wk8sz ns-xosbc Restart mysql-qzgper mysql Succeed 2/2 Jun 19,2025 18:40 UTC+0800 check ops status done ops_status:mysql-qzgper-restart-wk8sz ns-xosbc Restart mysql-qzgper mysql Succeed 2/2 Jun 19,2025 18:40 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-restart-wk8sz --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-restart-wk8sz patched `kbcli cluster delete-ops --name mysql-qzgper-restart-wk8sz --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-restart-wk8sz deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-1: out of mysql-qzgper-mysql-1(primary), mysql-qzgper-mysql-0(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover networkpartition check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkpartition-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-1 action: partition mode: all target: mode: all selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-0 direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkpartition-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkpartition-mysql-qzgper created apply test-chaos-mesh-networkpartition-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networkpartition-mysql-qzgper.yaml` networkpartition chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-mysql-qzgper" not found check failover pod name failover pod name:mysql-qzgper-mysql-1 failover networkpartition Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-1: out of mysql-qzgper-mysql-1(primary), mysql-qzgper-mysql-0(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover networklossover check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-1 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-mysql-qzgper created apply test-chaos-mesh-networklossover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networklossover-mysql-qzgper.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found check failover pod name failover pod name:mysql-qzgper-mysql-1 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-1 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-mysql-qzgper created apply test-chaos-mesh-networklossover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networklossover-mysql-qzgper.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found failover pod name:mysql-qzgper-mysql-1 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-1 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-mysql-qzgper created apply test-chaos-mesh-networklossover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networklossover-mysql-qzgper.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found failover pod name:mysql-qzgper-mysql-1 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-1 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-mysql-qzgper created apply test-chaos-mesh-networklossover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networklossover-mysql-qzgper.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-mysql-qzgper patched networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" force deleted failover pod name:mysql-qzgper-mysql-1 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-1 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-mysql-qzgper created apply test-chaos-mesh-networklossover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networklossover-mysql-qzgper.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-mysql-qzgper patched failover pod name:mysql-qzgper-mysql-1 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-1 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-mysql-qzgper created apply test-chaos-mesh-networklossover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networklossover-mysql-qzgper.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-qzgper" not found check failover pod name timeout test failover kill1 check node drain check node drain success `kill 1` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file exec return message: check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql NotReady secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check failover pod name failover pod name:mysql-qzgper-mysql-0 failover kill1 Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1:3306' (111) command terminated with exit code 1 checking cluster readonly data consistent... check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover dnsrandom check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mysql-qzgper" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnsrandom-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-0 mode: all action: random duration: 2m `kubectl apply -f test-chaos-mesh-dnsrandom-mysql-qzgper.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnsrandom-mysql-qzgper created apply test-chaos-mesh-dnsrandom-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-dnsrandom-mysql-qzgper.yaml` dnsrandom chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mysql-qzgper" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mysql-qzgper" not found check failover pod name failover pod name:mysql-qzgper-mysql-0 failover dnsrandom Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success cmpv upgrade service version:2,5.7.44|2,8.0.30|2,8.0.31|2,8.0.32|2,8.0.33|2,8.0.34|2,8.0.35|2,8.0.36|2,8.0.37|2,8.0.38|2,8.0.39|2,8.4.0|2,8.4.1|2,8.4.2 set latest cmpv service version latest service version:8.0.39 cmpv service version upgrade and downgrade 8.0.30 is less than 8.0.31 upgrade from:8.0.30 to service version:8.0.31 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.31 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-kbr45 created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-kbr45 ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:02 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-kbr45 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:02 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-kbr45 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:02 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-kbr45 --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-kbr45 patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-kbr45 --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-kbr45 deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... [Error] check cluster data consistent timeout `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1:3306' (111) command terminated with exit code 1 checking cluster readonly data consistent... check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success 8.0.31 is less than 8.0.32 upgrade from:8.0.31 to service version:8.0.32 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.32 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-lkwpg created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-lkwpg ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:05 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-lkwpg ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:05 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-lkwpg ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:05 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-lkwpg --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-lkwpg patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-lkwpg --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-lkwpg deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... [Error] check cluster data consistent timeout `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1:3306' (111) command terminated with exit code 1 checking cluster readonly data consistent... check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success 8.0.32 is less than 8.0.33 upgrade from:8.0.32 to service version:8.0.33 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.33 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-l8knc created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-l8knc ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:07 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-l8knc ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:07 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-l8knc ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:07 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-l8knc --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-l8knc patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-l8knc --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-l8knc deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... [Error] check cluster data consistent timeout `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1:3306' (111) command terminated with exit code 1 checking cluster readonly data consistent... check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success 8.0.33 is less than 8.0.34 upgrade from:8.0.33 to service version:8.0.34 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.34 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-r2kqz created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-r2kqz ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:10 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-r2kqz ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:10 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-r2kqz ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:10 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-r2kqz --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-r2kqz patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-r2kqz --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-r2kqz deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success upgrade from:8.0.34 to service version:8.0.35 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.35 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-wb4qm created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-wb4qm ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:12 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-wb4qm ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:12 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-wb4qm ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:12 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-wb4qm --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-wb4qm patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-wb4qm --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-wb4qm deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success upgrade from:8.0.35 to service version:8.0.39 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.39 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-dgzd5 created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-dgzd5 ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:15 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-dgzd5 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:15 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-dgzd5 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:15 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-dgzd5 --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-dgzd5 patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-dgzd5 --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-dgzd5 deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success downgrade from:8.0.39 to service version:8.0.35 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.35 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-s6fk7 created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-s6fk7 ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:17 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-s6fk7 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:17 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-s6fk7 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:17 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-s6fk7 --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-s6fk7 patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-s6fk7 --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-s6fk7 deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success upgrade from:8.0.35 to service version:8.0.36 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.36 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-szqb4 created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-szqb4 ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:18 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-szqb4 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:18 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-szqb4 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:18 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-szqb4 --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-szqb4 patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-szqb4 --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-szqb4 deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success upgrade from:8.0.36 to service version:8.0.39 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.39 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-hc476 created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-hc476 ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:20 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-hc476 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:20 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-hc476 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:20 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-hc476 --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-hc476 patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-hc476 --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-hc476 deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success downgrade from:8.0.39 to service version:8.0.36 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.36 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-ccnm5 created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-ccnm5 ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:22 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-ccnm5 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:22 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-ccnm5 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:22 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-ccnm5 --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-ccnm5 patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-ccnm5 --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-ccnm5 deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success upgrade from:8.0.36 to service version:8.0.37 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.37 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-9hkl4 created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-9hkl4 ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:24 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-9hkl4 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:24 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-9hkl4 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:24 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-9hkl4 --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-9hkl4 patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-9hkl4 --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-9hkl4 deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success upgrade from:8.0.37 to service version:8.0.39 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.39 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-g2kf6 created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-g2kf6 ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:26 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-g2kf6 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:26 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-g2kf6 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:26 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-g2kf6 --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-g2kf6 patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-g2kf6 --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-g2kf6 deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success downgrade from:8.0.39 to service version:8.0.37 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.37 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-kkqrh created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-kkqrh ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:28 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-kkqrh ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:28 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-kkqrh ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:28 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-kkqrh --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-kkqrh patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-kkqrh --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-kkqrh deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success upgrade from:8.0.37 to service version:8.0.38 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.38 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-l6595 created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-l6595 ns-xosbc Upgrade mysql-qzgper Creating -/- Jun 19,2025 19:29 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-l6595 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:29 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-l6595 ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:29 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-l6595 --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-l6595 patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-l6595 --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-l6595 deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success upgrade from:8.0.38 to service version:8.0.39 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.39 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-4b79s created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-4b79s ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:31 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-4b79s ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:31 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-4b79s ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:31 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-4b79s --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-4b79s patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-4b79s --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-4b79s deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success downgrade from:8.0.39 to service version:8.0.38 cluster upgrade apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-upgrade-cmpv- namespace: ns-xosbc spec: clusterName: mysql-qzgper upgrade: components: - componentName: mysql serviceVersion: 8.0.38 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-gj4hz created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-gj4hz ns-xosbc Upgrade mysql-qzgper mysql Running 0/2 Jun 19,2025 19:34 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-upgrade-cmpv-gj4hz ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:34 UTC+0800 check ops status done ops_status:mysql-qzgper-upgrade-cmpv-gj4hz ns-xosbc Upgrade mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:34 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-upgrade-cmpv-gj4hz --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-upgrade-cmpv-gj4hz patched `kbcli cluster delete-ops --name mysql-qzgper-upgrade-cmpv-gj4hz --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-upgrade-cmpv-gj4hz deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) command terminated with exit code 1 checking cluster data consistent... check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover networkcorruptover check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorruptover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-0 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-mysql-qzgper created apply test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml` networkcorruptover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found check failover pod name failover pod name:mysql-qzgper-mysql-0 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorruptover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-0 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-mysql-qzgper created apply test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml` networkcorruptover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found failover pod name:mysql-qzgper-mysql-0 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorruptover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-0 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-mysql-qzgper created apply test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml` networkcorruptover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found failover pod name:mysql-qzgper-mysql-0 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorruptover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-0 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-mysql-qzgper created apply test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml` networkcorruptover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found failover pod name:mysql-qzgper-mysql-0 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorruptover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-0 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-mysql-qzgper created apply test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml` networkcorruptover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found failover pod name:mysql-qzgper-mysql-0 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorruptover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-0 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-mysql-qzgper created apply test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networkcorruptover-mysql-qzgper.yaml` networkcorruptover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:42 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 18:40 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-qzgper" not found check failover pod name timeout cluster stop check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster stop mysql-qzgper --auto-approve --force=true --namespace ns-xosbc ` OpsRequest mysql-qzgper-stop-v97ft created successfully, you can view the progress: kbcli cluster describe-ops mysql-qzgper-stop-v97ft -n ns-xosbc check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-stop-v97ft ns-xosbc Stop mysql-qzgper mysql Running 0/2 Jun 19,2025 19:48 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Stopping Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Stopping check cluster status done cluster_status:Stopped check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME check pod status done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-stop-v97ft ns-xosbc Stop mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:48 UTC+0800 check ops status done ops_status:mysql-qzgper-stop-v97ft ns-xosbc Stop mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:48 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-stop-v97ft --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-stop-v97ft patched `kbcli cluster delete-ops --name mysql-qzgper-stop-v97ft --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-stop-v97ft deleted cluster start check cluster status before ops check cluster status done cluster_status:Stopped `kbcli cluster start mysql-qzgper --force=true --namespace ns-xosbc ` OpsRequest mysql-qzgper-start-gwtvz created successfully, you can view the progress: kbcli cluster describe-ops mysql-qzgper-start-gwtvz -n ns-xosbc check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-start-gwtvz ns-xosbc Start mysql-qzgper mysql Running 0/2 Jun 19,2025 19:49 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 19:49 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 19:49 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-start-gwtvz ns-xosbc Start mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:49 UTC+0800 check ops status done ops_status:mysql-qzgper-start-gwtvz ns-xosbc Start mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:49 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-start-gwtvz --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-start-gwtvz patched `kbcli cluster delete-ops --name mysql-qzgper-start-gwtvz --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-start-gwtvz deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover delete pod:mysql-qzgper-mysql-0 `kubectl delete pod mysql-qzgper-mysql-0 --namespace ns-xosbc ` pod "mysql-qzgper-mysql-0" deleted check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 19:51 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 19:49 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done check failover pod name failover pod name:mysql-qzgper-mysql-1 failover Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-1: out of mysql-qzgper-mysql-1(primary), mysql-qzgper-mysql-0(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover networkbandwidthover check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mysql-qzgper" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkbandwidthover-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-1 action: bandwidth mode: all bandwidth: rate: '1bps' limit: 20971520 buffer: 10000 duration: 2m `kubectl apply -f test-chaos-mesh-networkbandwidthover-mysql-qzgper.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkbandwidthover-mysql-qzgper created apply test-chaos-mesh-networkbandwidthover-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-networkbandwidthover-mysql-qzgper.yaml` networkbandwidthover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 19:51 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 19:49 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mysql-qzgper" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mysql-qzgper" not found check failover pod name failover pod name:mysql-qzgper-mysql-0 failover networkbandwidthover Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success `kubectl get pvc -l app.kubernetes.io/instance=mysql-qzgper,apps.kubeblocks.io/component-name=mysql,apps.kubeblocks.io/vct-name=data --namespace ns-xosbc ` cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in mysql-qzgper namespace. `kbcli cluster volume-expand mysql-qzgper --auto-approve --force=true --components mysql --volume-claim-templates data --storage 11Gi --namespace ns-xosbc ` OpsRequest mysql-qzgper-volumeexpansion-vnxw9 created successfully, you can view the progress: kbcli cluster describe-ops mysql-qzgper-volumeexpansion-vnxw9 -n ns-xosbc check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-volumeexpansion-vnxw9 ns-xosbc VolumeExpansion mysql-qzgper mysql Running 0/2 Jun 19,2025 19:55 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 19:51 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 19:49 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done No resources found in mysql-qzgper namespace. check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-volumeexpansion-vnxw9 ns-xosbc VolumeExpansion mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:55 UTC+0800 check ops status done ops_status:mysql-qzgper-volumeexpansion-vnxw9 ns-xosbc VolumeExpansion mysql-qzgper mysql Succeed 2/2 Jun 19,2025 19:55 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-volumeexpansion-vnxw9 --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-volumeexpansion-vnxw9 patched `kbcli cluster delete-ops --name mysql-qzgper-volumeexpansion-vnxw9 --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-volumeexpansion-vnxw9 deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test failover timeoffset check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mysql-qzgper" not found Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mysql-qzgper" not found apiVersion: chaos-mesh.org/v1alpha1 kind: TimeChaos metadata: name: test-chaos-mesh-timeoffset-mysql-qzgper namespace: ns-xosbc spec: selector: namespaces: - ns-xosbc labelSelectors: apps.kubeblocks.io/pod-name: mysql-qzgper-mysql-0 mode: all timeOffset: '-10m' clockIds: - CLOCK_REALTIME duration: 2m `kubectl apply -f test-chaos-mesh-timeoffset-mysql-qzgper.yaml` timechaos.chaos-mesh.org/test-chaos-mesh-timeoffset-mysql-qzgper created apply test-chaos-mesh-timeoffset-mysql-qzgper.yaml Success `rm -rf test-chaos-mesh-timeoffset-mysql-qzgper.yaml` timeoffset chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 19:51 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 19:49 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-mysql-qzgper --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mysql-qzgper" force deleted Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mysql-qzgper" not found check failover pod name failover pod name:mysql-qzgper-mysql-0 failover timeoffset Success `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success cluster hscale check cluster status before ops check cluster status done cluster_status:Running No resources found in mysql-qzgper namespace. `kbcli cluster hscale mysql-qzgper --auto-approve --force=true --components mysql --replicas 3 --namespace ns-xosbc ` OpsRequest mysql-qzgper-horizontalscaling-xtf9x created successfully, you can view the progress: kbcli cluster describe-ops mysql-qzgper-horizontalscaling-xtf9x -n ns-xosbc check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-horizontalscaling-xtf9x ns-xosbc HorizontalScaling mysql-qzgper mysql Running 0/1 Jun 19,2025 19:59 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 19:51 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 19:49 UTC+0800 mysql-qzgper-mysql-2 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 19:59 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 mysql-qzgper-mysql-2 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done No resources found in mysql-qzgper namespace. check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-horizontalscaling-xtf9x ns-xosbc HorizontalScaling mysql-qzgper mysql Succeed 1/1 Jun 19,2025 19:59 UTC+0800 check ops status done ops_status:mysql-qzgper-horizontalscaling-xtf9x ns-xosbc HorizontalScaling mysql-qzgper mysql Succeed 1/1 Jun 19,2025 19:59 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-horizontalscaling-xtf9x --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-horizontalscaling-xtf9x patched `kbcli cluster delete-ops --name mysql-qzgper-horizontalscaling-xtf9x --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-horizontalscaling-xtf9x deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary), mysql-qzgper-mysql-2(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success cluster hscale check cluster status before ops check cluster status done cluster_status:Running No resources found in mysql-qzgper namespace. `kbcli cluster hscale mysql-qzgper --auto-approve --force=true --components mysql --replicas 2 --namespace ns-xosbc ` OpsRequest mysql-qzgper-horizontalscaling-28bwr created successfully, you can view the progress: kbcli cluster describe-ops mysql-qzgper-horizontalscaling-28bwr -n ns-xosbc check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-horizontalscaling-28bwr ns-xosbc HorizontalScaling mysql-qzgper mysql Running 0/1 Jun 19,2025 20:01 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 19:51 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 19:49 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done No resources found in mysql-qzgper namespace. check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-horizontalscaling-28bwr ns-xosbc HorizontalScaling mysql-qzgper mysql Succeed 1/1 Jun 19,2025 20:01 UTC+0800 check ops status done ops_status:mysql-qzgper-horizontalscaling-28bwr ns-xosbc HorizontalScaling mysql-qzgper mysql Succeed 1/1 Jun 19,2025 20:01 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-horizontalscaling-28bwr --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-horizontalscaling-28bwr patched `kbcli cluster delete-ops --name mysql-qzgper-horizontalscaling-28bwr --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-horizontalscaling-28bwr deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success cluster configure component_tmp: mysql check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster configure mysql-qzgper --auto-approve --force=true --set binlog_expire_logs_seconds=691200 --components mysql --config-spec mysql-replication-config --config-file my.cnf --namespace ns-xosbc ` Will updated configure file meta: ConfigSpec: mysql-replication-config ConfigFile: my.cnf ComponentName: mysql ClusterName: mysql-qzgper OpsRequest mysql-qzgper-reconfiguring-g8xsq created successfully, you can view the progress: kbcli cluster describe-ops mysql-qzgper-reconfiguring-g8xsq -n ns-xosbc check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-reconfiguring-g8xsq ns-xosbc Reconfiguring mysql-qzgper mysql Running -/- Jun 19,2025 20:02 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 19:51 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 19:49 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-reconfiguring-g8xsq ns-xosbc Reconfiguring mysql-qzgper mysql Succeed 2/2 Jun 19,2025 20:02 UTC+0800 check ops status done ops_status:mysql-qzgper-reconfiguring-g8xsq ns-xosbc Reconfiguring mysql-qzgper mysql Succeed 2/2 Jun 19,2025 20:02 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-reconfiguring-g8xsq --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-reconfiguring-g8xsq patched `kbcli cluster delete-ops --name mysql-qzgper-reconfiguring-g8xsq --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-reconfiguring-g8xsq deleted check config variables Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. current value_actual: 691200 configure:[binlog_expire_logs_seconds] result actual:[691200] equal expected:[691200] `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success test switchover cluster promote check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster promote mysql-qzgper --auto-approve --force=true --component mysql --namespace ns-xosbc ` OpsRequest mysql-qzgper-custom-72vxh created successfully, you can view the progress: kbcli cluster describe-ops mysql-qzgper-custom-72vxh -n ns-xosbc check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-custom-72vxh ns-xosbc Custom mysql-qzgper mysql Running 0/1 Jun 19,2025 20:02 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 19:51 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:11Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 19:49 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-1;secondary: mysql-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-custom-72vxh ns-xosbc Custom mysql-qzgper mysql Succeed 1/1 Jun 19,2025 20:02 UTC+0800 check ops status done ops_status:mysql-qzgper-custom-72vxh ns-xosbc Custom mysql-qzgper mysql Succeed 1/1 Jun 19,2025 20:02 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-custom-72vxh --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-custom-72vxh patched `kbcli cluster delete-ops --name mysql-qzgper-custom-72vxh --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-custom-72vxh deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-1: out of mysql-qzgper-mysql-1(primary), mysql-qzgper-mysql-0(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success switchover pod:mysql-qzgper-mysql-1 switchover success No resources found in ns-xosbc namespace. `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql apiVersion: v1 kind: Pod metadata: name: benchtest-mysql-qzgper namespace: ns-xosbc spec: containers: - name: test-sysbench imagePullPolicy: IfNotPresent image: docker.io/apecloud/customsuites:latest env: - name: TYPE value: "2" - name: FLAG value: "0" - name: CONFIGS value: "mode:all,driver:mysql,host:mysql-qzgper-mysql-mysql-server.ns-xosbc.svc.cluster.local,user:root,password:czo40138CR,port:3306,db:benchtest,tables:5,threads:4,times:10,size:1000,type:oltp_read_write" restartPolicy: Never `kubectl apply -f benchtest-mysql-qzgper.yaml` pod/benchtest-mysql-qzgper created apply benchtest-mysql-qzgper.yaml Success `rm -rf benchtest-mysql-qzgper.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 0/1 ContainerCreating 0 0s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 0/1 ContainerCreating 0 5s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 0/1 ContainerCreating 0 11s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 0/1 ContainerCreating 0 17s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 0/1 ContainerCreating 0 22s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 1/1 Running 0 28s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 1/1 Running 0 34s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 1/1 Running 0 40s check pod benchtest-mysql-qzgper status done pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 0/1 Completed 0 46s `kubectl logs benchtest-mysql-qzgper --tail 30 --namespace ns-xosbc ` [ 7s ] thds: 4 tps: 10.00 qps: 216.01 (r/w/o: 149.01/47.00/20.00) lat (ms,99%): 590.56 err/s: 0.00 reconn/s: 0.00 [ 8s ] thds: 4 tps: 10.00 qps: 197.00 (r/w/o: 136.00/41.00/20.00) lat (ms,99%): 601.29 err/s: 0.00 reconn/s: 0.00 [ 9s ] thds: 4 tps: 10.00 qps: 201.98 (r/w/o: 140.99/41.00/20.00) lat (ms,99%): 694.45 err/s: 0.00 reconn/s: 0.00 [ 10s ] thds: 4 tps: 11.00 qps: 225.02 (r/w/o: 157.01/46.00/22.00) lat (ms,99%): 502.20 err/s: 0.00 reconn/s: 0.00 SQL statistics: queries performed: read: 1526 write: 436 other: 218 total: 2180 transactions: 109 (10.78 per sec.) queries: 2180 (215.68 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 10.1062s total number of events: 109 Latency (ms): min: 102.43 avg: 370.63 max: 695.54 99th percentile: 601.29 sum: 40398.17 Threads fairness: events (avg/stddev): 27.2500/0.83 execution time (avg/stddev): 10.0995/0.00 `kubectl delete pod benchtest-mysql-qzgper --force --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "benchtest-mysql-qzgper" force deleted No resources found in ns-xosbc namespace. `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql apiVersion: v1 kind: Pod metadata: name: benchtest-mysql-qzgper namespace: ns-xosbc spec: containers: - name: test-sysbench imagePullPolicy: IfNotPresent image: docker.io/apecloud/customsuites:latest env: - name: TYPE value: "2" - name: FLAG value: "0" - name: CONFIGS value: "mode:all,driver:mysql,host:ac5f1b69237d942ceb8b0ea135c55bc7-69ec54396292e72c.elb.us-west-2.amazonaws.com,user:root,password:czo40138CR,port:3306,db:benchtest,tables:5,threads:4,times:10,size:1000,type:oltp_read_write" restartPolicy: Never `kubectl apply -f benchtest-mysql-qzgper.yaml` pod/benchtest-mysql-qzgper created apply benchtest-mysql-qzgper.yaml Success `rm -rf benchtest-mysql-qzgper.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 1/1 Running 0 0s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 1/1 Running 0 5s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 1/1 Running 0 11s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 1/1 Running 0 17s check pod benchtest-mysql-qzgper status done pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-qzgper 0/1 Completed 0 22s `kubectl logs benchtest-mysql-qzgper --tail 30 --namespace ns-xosbc ` [ 7s ] thds: 4 tps: 10.00 qps: 200.99 (r/w/o: 143.99/36.00/21.00) lat (ms,99%): 694.45 err/s: 0.00 reconn/s: 0.00 [ 8s ] thds: 4 tps: 10.00 qps: 194.02 (r/w/o: 132.01/43.00/19.00) lat (ms,99%): 694.45 err/s: 0.00 reconn/s: 0.00 [ 9s ] thds: 4 tps: 11.00 qps: 240.99 (r/w/o: 169.99/48.00/23.00) lat (ms,99%): 502.20 err/s: 0.00 reconn/s: 0.00 [ 10s ] thds: 4 tps: 12.00 qps: 221.01 (r/w/o: 160.01/37.00/24.00) lat (ms,99%): 404.61 err/s: 0.00 reconn/s: 0.00 SQL statistics: queries performed: read: 1554 write: 444 other: 222 total: 2220 transactions: 111 (10.78 per sec.) queries: 2220 (215.57 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 10.2969s total number of events: 111 Latency (ms): min: 105.50 avg: 368.31 max: 696.30 99th percentile: 694.45 sum: 40882.29 Threads fairness: events (avg/stddev): 27.7500/1.09 execution time (avg/stddev): 10.2206/0.04 `kubectl delete pod benchtest-mysql-qzgper --force --namespace ns-xosbc ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "benchtest-mysql-qzgper" force deleted cluster vscale check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale mysql-qzgper --auto-approve --force=true --components mysql --cpu 200m --memory 0.6Gi --namespace ns-xosbc ` OpsRequest mysql-qzgper-verticalscaling-69vnj created successfully, you can view the progress: kbcli cluster describe-ops mysql-qzgper-verticalscaling-69vnj -n ns-xosbc check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-verticalscaling-69vnj ns-xosbc VerticalScaling mysql-qzgper mysql Running 0/2 Jun 19,2025 20:07 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc Delete Updating Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:11Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 20:07 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:11Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 20:08 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-verticalscaling-69vnj ns-xosbc VerticalScaling mysql-qzgper mysql Succeed 2/2 Jun 19,2025 20:07 UTC+0800 check ops status done ops_status:mysql-qzgper-verticalscaling-69vnj ns-xosbc VerticalScaling mysql-qzgper mysql Succeed 2/2 Jun 19,2025 20:07 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-verticalscaling-69vnj --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-verticalscaling-69vnj patched `kbcli cluster delete-ops --name mysql-qzgper-verticalscaling-69vnj --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-verticalscaling-69vnj deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success cluster update terminationPolicy WipeOut `kbcli cluster update mysql-qzgper --termination-policy=WipeOut --namespace ns-xosbc ` cluster.apps.kubeblocks.io/mysql-qzgper updated check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc WipeOut Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:11Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 20:07 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:11Gi ip-172-31-10-206.us-west-2.compute.internal/172.31.10.206 Jun 19,2025 20:08 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done cluster xtrabackup backup `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.name***"` `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.namespace***"` `kubectl get secrets kb-backuprepo-w7bzj -n kb-mqtcr -o jsonpath="***.data.accessKeyId***"` `kubectl get secrets kb-backuprepo-w7bzj -n kb-mqtcr -o jsonpath="***.data.secretAccessKey***"` KUBEBLOCKS NAMESPACE:kb-mqtcr get kubeblocks namespace done `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-mqtcr -o jsonpath="***.items[0].data.root-user***"` `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-mqtcr -o jsonpath="***.items[0].data.root-password***"` minio_user:kbclitest,minio_password:kbclitest,minio_endpoint:kbcli-test-minio.kb-mqtcr.svc.cluster.local:9000 list minio bucket kbcli-test `echo 'mc config host add minioserver http://kbcli-test-minio.kb-mqtcr.svc.cluster.local:9000 kbclitest kbclitest;mc ls minioserver' | kubectl exec -it kbcli-test-minio-5c88d6b598-9v8l5 --namespace kb-mqtcr -- bash` Unable to use a TTY - input is not a terminal or the right kind of file list minio bucket done default backuprepo:backuprepo-kbcli-test exists `kbcli cluster backup mysql-qzgper --method xtrabackup --namespace ns-xosbc ` Backup backup-ns-xosbc-mysql-qzgper-20250619200934 created successfully, you can view the progress: kbcli cluster list-backups --name=backup-ns-xosbc-mysql-qzgper-20250619200934 -n ns-xosbc check backup status `kbcli cluster list-backups mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION CREATE-TIME COMPLETION-TIME EXPIRATION backup-ns-xosbc-mysql-qzgper-20250619200934 ns-xosbc mysql-qzgper xtrabackup Running Jun 19,2025 20:09 UTC+0800 backup_status:mysql-qzgper-xtrabackup-Running backup_status:mysql-qzgper-xtrabackup-Running backup_status:mysql-qzgper-xtrabackup-Running backup_status:mysql-qzgper-xtrabackup-Running check backup status done backup_status:backup-ns-xosbc-mysql-qzgper-20250619200934 ns-xosbc mysql-qzgper xtrabackup Completed 6178166 19s Jun 19,2025 20:09 UTC+0800 Jun 19,2025 20:09 UTC+0800 cluster restore backup Error from server (NotFound): opsrequests.apps.kubeblocks.io "abk-qzgper" not found `kbcli cluster describe-backup backup-ns-xosbc-mysql-qzgper-20250619200934 --namespace ns-xosbc ` Name: backup-ns-xosbc-mysql-qzgper-20250619200934 Cluster: mysql-qzgper Namespace: ns-xosbc Spec: Method: xtrabackup Policy Name: mysql-qzgper-mysql-backup-policy Status: Phase: Completed Total Size: 6178166 ActionSet Name: mysql-xtrabackup Repository: backuprepo-kbcli-test Duration: 19s Start Time: Jun 19,2025 20:09 UTC+0800 Completion Time: Jun 19,2025 20:09 UTC+0800 Path: /ns-xosbc/mysql-qzgper-f11d251c-9bc9-4903-8859-68f9b2f98ba2/mysql/backup-ns-xosbc-mysql-qzgper-20250619200934 Warning Events: `kbcli cluster restore abk-qzgper --backup backup-ns-xosbc-mysql-qzgper-20250619200934 --namespace ns-xosbc ` Cluster abk-qzgper created check cluster status `kbcli cluster list abk-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS abk-qzgper ns-xosbc WipeOut Jun 19,2025 20:10 UTC+0800 cluster_status: cluster_status: cluster_status: cluster_status: cluster_status: cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances abk-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME abk-qzgper-mysql-0 ns-xosbc abk-qzgper mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:11Gi ip-172-31-14-205.us-west-2.compute.internal/172.31.14.205 Jun 19,2025 20:10 UTC+0800 abk-qzgper-mysql-1 ns-xosbc abk-qzgper mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:11Gi ip-172-31-5-129.us-west-2.compute.internal/172.31.5.129 Jun 19,2025 20:10 UTC+0800 check pod status done check cluster role check cluster role done primary: abk-qzgper-mysql-1;secondary: abk-qzgper-mysql-0 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i abk-qzgper-mysql-0 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `kbcli cluster describe-backup backup-ns-xosbc-mysql-qzgper-20250619200934 --namespace ns-xosbc ` Name: backup-ns-xosbc-mysql-qzgper-20250619200934 Cluster: mysql-qzgper Namespace: ns-xosbc Spec: Method: xtrabackup Policy Name: mysql-qzgper-mysql-backup-policy Status: Phase: Completed Total Size: 6178166 ActionSet Name: mysql-xtrabackup Repository: backuprepo-kbcli-test Duration: 19s Start Time: Jun 19,2025 20:09 UTC+0800 Completion Time: Jun 19,2025 20:09 UTC+0800 Path: /ns-xosbc/mysql-qzgper-f11d251c-9bc9-4903-8859-68f9b2f98ba2/mysql/backup-ns-xosbc-mysql-qzgper-20250619200934 Warning Events: cluster connect Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. *************************** 1. row *************************** Replica_IO_State: Waiting for source to send event Source_Host: abk-qzgper-mysql-1.abk-qzgper-mysql-headless Source_User: root Source_Port: 3306 Connect_Retry: 60 Source_Log_File: abk-qzgper-mysql-1-bin.000001 Read_Source_Log_Pos: 58060 Relay_Log_File: relay-bin.000002 Relay_Log_Pos: 25662 Relay_Source_Log_File: abk-qzgper-mysql-1-bin.000001 Replica_IO_Running: Yes Replica_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Source_Log_Pos: 58060 Relay_Log_Space: 25866 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Source_SSL_Allowed: Yes Source_SSL_CA_File: Source_SSL_CA_Path: Source_SSL_Cert: Source_SSL_Cipher: Source_SSL_Key: Seconds_Behind_Source: 0 Source_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Source_Server_Id: 2 Source_UUID: 717de74b-4d06-11f0-9da3-5e7739162b3d Source_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Replica_SQL_Running_State: Replica has read all relay log; waiting for more updates Source_Retry_Count: 86400 Source_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Source_SSL_Crl: Source_SSL_Crlpath: Retrieved_Gtid_Set: 717de74b-4d06-11f0-9da3-5e7739162b3d:1-63 Executed_Gtid_Set: 717de74b-4d06-11f0-9da3-5e7739162b3d:1-63, ba378b06-4cf6-11f0-8d35-3a0c1a099b0d:1-3109, be7d55a2-4cf6-11f0-abf1-127debd9f66d:1-2190 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Source_TLS_Version: Source_public_key_path: Get_Source_public_key: 0 Network_Namespace: `echo 'show variables like "max_connections";' | kbcli cluster connect abk-qzgper --namespace ns-xosbc ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Connect to instance abk-qzgper-mysql-1: out of abk-qzgper-mysql-1(primary), abk-qzgper-mysql-0(secondary) Variable_name Value max_connections 100 connect cluster Success set max_connections to 100 delete cluster abk-qzgper `kbcli cluster delete abk-qzgper --auto-approve --namespace ns-xosbc ` Cluster abk-qzgper deleted pod_info:abk-qzgper-mysql-0 3/3 Running 0 97s abk-qzgper-mysql-1 3/3 Running 0 97s No resources found in ns-xosbc namespace. delete cluster pod done No resources found in ns-xosbc namespace. check cluster resource non-exist OK: pvc No resources found in ns-xosbc namespace. delete cluster done No resources found in ns-xosbc namespace. No resources found in ns-xosbc namespace. No resources found in ns-xosbc namespace. cluster rebulid instances apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-qzgper-rebuildinstance- namespace: ns-xosbc spec: type: RebuildInstance clusterName: mysql-qzgper force: true rebuildFrom: - componentName: mysql instances: - name: mysql-qzgper-mysql-1 backupName: backup-ns-xosbc-mysql-qzgper-20250619200934 inPlace: true restoreEnv: - name: BACKUP_FOR_STANDBY value: "true" check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-qzgper.yaml` opsrequest.apps.kubeblocks.io/mysql-qzgper-rebuildinstance-5scrb created create test_ops_cluster_mysql-qzgper.yaml Success `rm -rf test_ops_cluster_mysql-qzgper.yaml` check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Running 0/1 Jun 19,2025 20:12 UTC+0800 check ops status done ops_status:mysql-qzgper-rebuildinstance-5scrb ns-xosbc RebuildInstance mysql-qzgper mysql Succeed 1/1 Jun 19,2025 20:12 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-rebuildinstance-5scrb --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-rebuildinstance-5scrb patched `kbcli cluster delete-ops --name mysql-qzgper-rebuildinstance-5scrb --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-rebuildinstance-5scrb deleted check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc WipeOut Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:11Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 20:07 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:11Gi ip-172-31-14-205.us-west-2.compute.internal/172.31.14.205 Jun 19,2025 20:12 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups backup-ns-xosbc-mysql-qzgper-20250619200934 --namespace ns-xosbc ` backup.dataprotection.kubeblocks.io/backup-ns-xosbc-mysql-qzgper-20250619200934 patched `kbcli cluster delete-backup mysql-qzgper --name backup-ns-xosbc-mysql-qzgper-20250619200934 --force --auto-approve --namespace ns-xosbc ` Backup backup-ns-xosbc-mysql-qzgper-20250619200934 deleted No opsrequests found in ns-xosbc namespace. `kubectl get backupschedule -l app.kubernetes.io/instance=mysql-qzgper ` No resources found in ns-xosbc namespace. `kubectl get backupschedule -l app.kubernetes.io/instance=mysql-qzgper ` No resources found in ns-xosbc namespace. `kubectl get backupschedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` cluster list-logs `kbcli cluster list-logs mysql-qzgper --namespace ns-xosbc ` No log files found. You can enable the log feature with the kbcli command below. kbcli cluster update mysql-qzgper --enable-all-logs=true --namespace ns-xosbc Error from server (NotFound): pods "mysql-qzgper-mysql-0" not found cluster logs `kbcli cluster logs mysql-qzgper --tail 30 --namespace ns-xosbc ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) 2025-06-19T12:07:56Z INFO MySQL switch to root db == DB == 2025-06-19 12:07:56+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.38-1.el9 started. == DB == 2025-06-19 12:07:58+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql' == DB == 2025-06-19 12:07:58+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.38-1.el9 started. == DB == '/var/lib/mysql/mysql.sock' -> '/var/run/mysqld/mysqld.sock' 2025-06-19T12:08:01Z INFO HA wait for db service to be ready 2025-06-19T12:08:01Z INFO MySQL DB is not ready and try to reconnect ***"error": "dial tcp 127.0.0.1:3306: connect: connection refused"*** 2025-06-19T12:08:01Z INFO MySQL switch to root db 2025-06-19T12:08:06Z INFO HA wait for db service to be ready 2025-06-19T12:08:06Z INFO MySQL exec: start replica; 2025-06-19T12:08:06Z INFO MySQL Set semi-sync source timeout ***"sql": "SET GLOBAL rpl_semi_sync_source_timeout = 10000;", "leader": "mysql-qzgper-mysql-1"*** 2025-06-19T12:08:06Z INFO MySQL set semi-sync source timeout failed ***"error": "SET GLOBAL rpl_semi_sync_source_timeout = 10000; execute failed: dial tcp 172.31.5.253:3306: connect: connection refused"*** 2025-06-19T12:08:06Z INFO HA DB Service is running 2025-06-19T12:08:06Z INFO HA Cluster has no leader, attempt to take the leader 2025-06-19T12:08:06Z INFO MySQL get op timestamp failed ***"error": "dial tcp 172.31.5.253:3306: connect: connection refused"*** 2025-06-19T12:08:06Z INFO HA Take the leader success! 2025-06-19T12:08:06Z INFO HA This member is Cluster's leader 2025-06-19T12:08:06Z DEBUG HA Refresh leader ttl 2025-06-19T12:09:06Z INFO HA This member is Cluster's leader 2025-06-19T12:09:06Z DEBUG HA Refresh leader ttl 2025-06-19T12:10:06Z INFO HA This member is Cluster's leader 2025-06-19T12:10:06Z DEBUG HA Refresh leader ttl 2025-06-19T12:11:06Z INFO HA This member is Cluster's leader 2025-06-19T12:11:06Z DEBUG HA Refresh leader ttl 2025-06-19T12:12:06Z INFO HA This member is Cluster's leader 2025-06-19T12:12:06Z DEBUG HA Refresh leader ttl 2025-06-19T12:13:06Z INFO HA This member is Cluster's leader 2025-06-19T12:13:06Z DEBUG HA Refresh leader ttl 2025-06-19T12:14:06Z INFO HA This member is Cluster's leader 2025-06-19T12:14:06Z DEBUG HA Refresh leader ttl cluster logs slow `kbcli cluster logs mysql-qzgper --tail 30 --file-type=slow --namespace ns-xosbc ` error: get pod component definition name in cluster.yaml fail cluster logs error `kbcli cluster logs mysql-qzgper --tail 30 --file-type=error --namespace ns-xosbc ` error: get pod component definition name in cluster.yaml fail cluster expose check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster expose mysql-qzgper --auto-approve --force=true --type internet --enable false --components mysql --namespace ns-xosbc ` OpsRequest mysql-qzgper-expose-w89r4 created successfully, you can view the progress: kbcli cluster describe-ops mysql-qzgper-expose-w89r4 -n ns-xosbc check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-expose-w89r4 ns-xosbc Expose mysql-qzgper mysql Running 0/1 Jun 19,2025 20:14 UTC+0800 check cluster status `kbcli cluster list mysql-qzgper --show-labels --namespace ns-xosbc ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-qzgper ns-xosbc WipeOut Running Jun 19,2025 18:16 UTC+0800 app.kubernetes.io/instance=mysql-qzgper check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-qzgper --namespace ns-xosbc ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-qzgper-mysql-0 ns-xosbc mysql-qzgper mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:11Gi ip-172-31-7-160.us-west-2.compute.internal/172.31.7.160 Jun 19,2025 20:07 UTC+0800 mysql-qzgper-mysql-1 ns-xosbc mysql-qzgper mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:11Gi ip-172-31-14-205.us-west-2.compute.internal/172.31.14.205 Jun 19,2025 20:12 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-qzgper-mysql-0;secondary: mysql-qzgper-mysql-1 check cluster connect `echo 'SHOW REPLICA STATUS\G;' | kbcli cluster connect --namespace ns-xosbc -i mysql-qzgper-mysql-1 ` Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. check mysql slave success check cluster connect done check ops status `kbcli cluster list-ops mysql-qzgper --status all --namespace ns-xosbc ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-qzgper-expose-w89r4 ns-xosbc Expose mysql-qzgper mysql Succeed 1/1 Jun 19,2025 20:14 UTC+0800 check ops status done ops_status:mysql-qzgper-expose-w89r4 ns-xosbc Expose mysql-qzgper mysql Succeed 1/1 Jun 19,2025 20:14 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mysql-qzgper-expose-w89r4 --namespace ns-xosbc ` opsrequest.apps.kubeblocks.io/mysql-qzgper-expose-w89r4 patched `kbcli cluster delete-ops --name mysql-qzgper-expose-w89r4 --force --auto-approve --namespace ns-xosbc ` OpsRequest mysql-qzgper-expose-w89r4 deleted `echo 'use mydb; SELECT value FROM tmp_table WHERE id = 1;' | kbcli cluster connect mysql-qzgper --namespace ns-xosbc ` check data: Connect to instance mysql-qzgper-mysql-0: out of mysql-qzgper-mysql-0(primary), mysql-qzgper-mysql-1(secondary) Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql `echo "echo \"use mydb; SELECT value FROM tmp_table WHERE id = 1;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR'" | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly data: Defaulted container "mysql" out of: mysql, lorry, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. value llsap check cluster readonly data consistent Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-qzgper-mysql-mysql-0.ns-xosbc.svc.cluster.local -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-0 --namespace ns-xosbc -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-qzgper` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-qzgper-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:czo40138CR;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'czo40138CR' " | kubectl exec -it mysql-qzgper-mysql-1 --namespace ns-xosbc -- bash ` check readonly db_client batch data Success delete cluster mysql-qzgper `kbcli cluster delete mysql-qzgper --auto-approve --namespace ns-xosbc ` Cluster mysql-qzgper deleted pod_info:mysql-qzgper-mysql-0 3/3 Running 0 7m45s mysql-qzgper-mysql-1 3/3 Running 0 2m31s No resources found in ns-xosbc namespace. delete cluster pod done No resources found in ns-xosbc namespace. check cluster resource non-exist OK: pvc No resources found in ns-xosbc namespace. delete cluster done No resources found in ns-xosbc namespace. No resources found in ns-xosbc namespace. No resources found in ns-xosbc namespace. Mysql Test Suite All Done! --------------------------------------Mysql (Topology = Replicas 2) Test Result-------------------------------------- [PASSED]|[Create]|[ComponentVersion=mysql;ServiceVersion=8.0.30;]|[Description=Create a cluster and component version mysql and service version 8.0.30] [PASSED]|[Connect]|[ComponentName=mysql]|[Description=Connect to the cluster] [PASSED]|[AddData]|[Values=llsap]|[Description=Add data to the cluster] [PASSED]|[CheckAddDataReadonly]|[Values=llsap;Role=Readonly]|[Description=Add data to the cluster readonly] [PASSED]|[Expose]|[Enable=true;TYPE=internet;ComponentName=mysql]|[Description=Expose Enable the internet service with mysql component] [PASSED]|[Failover]|[HA=OOM;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience OOM either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Memory load.] [PASSED]|[Failover]|[HA=DNS Error;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience DNS service errors for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to DNS service errors.] [PASSED]|[Check Monitor]|[Enable=true]|[Description=Check Monitor Enable] [PASSED]|[Update]|[Monitor=true]|[Description=Update the cluster Monitor enable] [PASSED]|[Failover]|[HA=Network Duplicate;Durations=2m;ComponentName=mysql]|[Description=Simulates network duplicate fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to duplicate network.] [PASSED]|[Failover]|[HA=Full CPU;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience CPU full either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high CPU load.] [PASSED]|[Failover]|[HA=Evicting Pod;ComponentName=mysql]|[Description=Simulates conditions where pods evicting either due to node drained thereby testing the application's resilience to unavailability of some replicas due to evicting.] [PASSED]|[Failover]|[HA=Connection Stress;ComponentName=mysql]|[Description=Simulates conditions where pods experience connection stress either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Connection load.] [PASSED]|[Update]|[Logs=Enable]|[Description=Update the cluster Logs enable] [PASSED]|[Failover]|[HA=Network Delay;Durations=2m;ComponentName=mysql]|[Description=Simulates network delay fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to delay network.] [PASSED]|[Failover]|[HA=Pod Failure;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience failure for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to failure.] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [PASSED]|[Failover]|[HA=Network Partition;Durations=2m;ComponentName=mysql]|[Description=Simulates network partition fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to partition network.] [FAILED]|[Failover]|[HA=Network Loss Failover;Durations=2m;ComponentName=mysql]|[Description=Simulates network loss fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to loss network.] [PASSED]|[Failover]|[HA=Kill 1;ComponentName=mysql]|[Description=Simulates conditions where process 1 killed either due to expected/undesired processes thereby testing the application's resilience to unavailability of some replicas due to abnormal termination signals.] [PASSED]|[Failover]|[HA=DNS Random;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience random IP addresses being returned by the DNS service for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to the DNS service returning random IP addresses.] [FAILED]|[CheckData]|[OpsType=Upgrade]|[Description=Check the cluster data consistent] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.30;ComponentVersionTo=8.0.31]|[Description=Upgrade the cluster specify component mysql service version from 8.0.30 to 8.0.31] [FAILED]|[CheckData]|[OpsType=Upgrade]|[Description=Check the cluster data consistent] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.31;ComponentVersionTo=8.0.32]|[Description=Upgrade the cluster specify component mysql service version from 8.0.31 to 8.0.32] [FAILED]|[CheckData]|[OpsType=Upgrade]|[Description=Check the cluster data consistent] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.32;ComponentVersionTo=8.0.33]|[Description=Upgrade the cluster specify component mysql service version from 8.0.32 to 8.0.33] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.33;ComponentVersionTo=8.0.34]|[Description=Upgrade the cluster specify component mysql service version from 8.0.33 to 8.0.34] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.34;ComponentVersionTo=8.0.35]|[Description=Upgrade the cluster specify component mysql service version from 8.0.34 to 8.0.35] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.35;ComponentVersionTo=8.0.39]|[Description=Upgrade the cluster specify component mysql service version from 8.0.35 to 8.0.39] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.39;ComponentVersionTo=8.0.35]|[Description=Upgrade the cluster specify component mysql service version from 8.0.39 to 8.0.35] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.35;ComponentVersionTo=8.0.36]|[Description=Upgrade the cluster specify component mysql service version from 8.0.35 to 8.0.36] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.36;ComponentVersionTo=8.0.39]|[Description=Upgrade the cluster specify component mysql service version from 8.0.36 to 8.0.39] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.39;ComponentVersionTo=8.0.36]|[Description=Upgrade the cluster specify component mysql service version from 8.0.39 to 8.0.36] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.36;ComponentVersionTo=8.0.37]|[Description=Upgrade the cluster specify component mysql service version from 8.0.36 to 8.0.37] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.37;ComponentVersionTo=8.0.39]|[Description=Upgrade the cluster specify component mysql service version from 8.0.37 to 8.0.39] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.39;ComponentVersionTo=8.0.37]|[Description=Upgrade the cluster specify component mysql service version from 8.0.39 to 8.0.37] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.37;ComponentVersionTo=8.0.38]|[Description=Upgrade the cluster specify component mysql service version from 8.0.37 to 8.0.38] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.38;ComponentVersionTo=8.0.39]|[Description=Upgrade the cluster specify component mysql service version from 8.0.38 to 8.0.39] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.39;ComponentVersionTo=8.0.38]|[Description=Upgrade the cluster specify component mysql service version from 8.0.39 to 8.0.38] [FAILED]|[Failover]|[HA=Network Corrupt Failover;Durations=2m;ComponentName=mysql]|[Description=Simulates network corrupt fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to corrupt network.] [PASSED]|[Stop]|[-]|[Description=Stop the cluster] [PASSED]|[Start]|[-]|[Description=Start the cluster] [PASSED]|[Failover]|[HA=Delete Pod;ComponentName=mysql]|[Description=Simulates conditions where pods terminating forced/graceful thereby testing deployment sanity (replica availability & uninterrupted service) and recovery workflow of the application.] [PASSED]|[Failover]|[HA=Network Bandwidth Failover;Durations=2m;ComponentName=mysql]|[Description=Simulates network bandwidth fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to bandwidth network.] [PASSED]|[VolumeExpansion]|[ComponentName=mysql]|[Description=VolumeExpansion the cluster specify component mysql] [PASSED]|[Failover]|[HA=Time Offset;Durations=2m;ComponentName=mysql]|[Description=Simulates a time offset scenario thereby testing the application's resilience to potential slowness/unavailability of some replicas due to time offset.] [PASSED]|[HorizontalScaling Out]|[ComponentName=mysql]|[Description=HorizontalScaling Out the cluster specify component mysql] [PASSED]|[HorizontalScaling In]|[ComponentName=mysql]|[Description=HorizontalScaling In the cluster specify component mysql] [PASSED]|[Reconfiguring]|[ComponentName=mysql;binlog_expire_logs_seconds=691200]|[Description=Reconfiguring the cluster specify component mysql set binlog_expire_logs_seconds=691200] [PASSED]|[SwitchOver]|[ComponentName=mysql]|[Description=SwitchOver the cluster specify component mysql] [PASSED]|[Bench]|[ComponentName=mysql]|[Description=Bench the cluster service with mysql component] [PASSED]|[Bench]|[HostType=LB;ComponentName=mysql]|[Description=Bench the cluster LB service with mysql component] [PASSED]|[VerticalScaling]|[ComponentName=mysql]|[Description=VerticalScaling the cluster specify component mysql] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Backup]|[BackupMethod=xtrabackup]|[Description=The cluster xtrabackup Backup] [PASSED]|[Restore]|[BackupMethod=xtrabackup]|[Description=The cluster xtrabackup Restore] [PASSED]|[Connect]|[ComponentName=mysql]|[Description=Connect to the cluster] [PASSED]|[Delete Restore Cluster]|[BackupMethod=xtrabackup]|[Description=Delete the xtrabackup restore cluster] [PASSED]|[RebuildInstance]|[ComponentName=mysql]|[Description=Rebuild the cluster instance specify component mysql] [PASSED]|[Expose]|[Disable=true;TYPE=internet;ComponentName=mysql]|[Description=Expose Disable the internet service with mysql component] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]