source commons files source engines files source kubeblocks files `kubectl get namespace | grep ns-lweur ` `kubectl create namespace ns-lweur` namespace/ns-lweur created create namespace ns-lweur done download kbcli `gh release list --repo apecloud/kbcli --limit 100 | (grep "1.0" || true)` `curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v1.0.0` Your system is linux_amd64 Installing kbcli ... Downloading ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 33.6M 100 33.6M 0 0 81.7M 0 --:--:-- --:--:-- --:--:-- 81.7M kbcli installed successfully. Kubernetes: v1.32.5-eks-5d4a308 KubeBlocks: 1.0.0 kbcli: 1.0.0 Make sure your docker service is running and begin your journey with kbcli: kbcli playground init For more information on how to get started, please visit: https://kubeblocks.io download kbcli v1.0.0 done Kubernetes: v1.32.5-eks-5d4a308 KubeBlocks: 1.0.0 kbcli: 1.0.0 Kubernetes Env: v1.32.5-eks-5d4a308 check snapshot controller check snapshot controller done eks default-vsc found No resources found POD_RESOURCES: found default storage class: gp3 KubeBlocks version is:1.0.0 skip upgrade KubeBlocks current KubeBlocks version: 1.0.0 Error: no repositories to show helm repo add chaos-mesh https://charts.chaos-mesh.org "chaos-mesh" has been added to your repositories add helm chart repo chaos-mesh success chaos mesh already installed check component definition set component name:mysql set component version set component version:mysql set service versions:8.4.2,8.4.1,8.4.0,8.0.39,8.0.38,8.0.37,8.0.36,8.0.35,8.0.34,8.0.33,8.0.32,8.0.31,8.0.30,5.7.44 set service versions sorted:5.7.44,8.0.30,8.0.31,8.0.32,8.0.33,8.0.34,8.0.35,8.0.36,8.0.37,8.0.38,8.0.39,8.4.0,8.4.1,8.4.2 set mysql component definition set mysql component definition set replicas first:2,5.7.44|2,8.0.30|2,8.0.31|2,8.0.32|2,8.0.33|2,8.0.34|2,8.0.35|2,8.0.36|2,8.0.37|2,8.0.38|2,8.0.39|2,8.4.0|2,8.4.1|2,8.4.2 set replicas third:2,8.0.35 set replicas fourth:2,8.0.30 set minimum cmpv service version set minimum cmpv service version replicas:2,8.0.30 REPORT_COUNT:1 CLUSTER_TOPOLOGY:semisync-proxysql topology semisync-proxysql found in cluster definition mysql set mysql component definition set mysql component definition LIMIT_CPU:0.1 LIMIT_MEMORY:0.5 storage size: 10 No resources found in ns-lweur namespace. termination_policy:Delete create 2 replica Delete mysql cluster check component definition set component definition by component version check cmpd by labels set component definition1: mysql-8.0-1.0.0-alpha.0 by component version:mysql apiVersion: apps.kubeblocks.io/v1 kind: Cluster metadata: name: mysql-niyrwp namespace: ns-lweur spec: clusterDef: mysql topology: semisync-proxysql terminationPolicy: Delete componentSpecs: - name: mysql serviceVersion: 8.0.30 env: - name: PROXY_ENABLED value: "on" disableExporter: true replicas: 2 resources: limits: cpu: 100m memory: 0.5Gi requests: cpu: 100m memory: 0.5Gi volumeClaimTemplates: - name: data spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi - name: proxysql replicas: 1 resources: limits: cpu: 100m memory: 0.5Gi requests: cpu: 100m memory: 0.5Gi `kubectl apply -f test_create_mysql-niyrwp.yaml` cluster.apps.kubeblocks.io/mysql-niyrwp created apply test_create_mysql-niyrwp.yaml Success `rm -rf test_create_mysql-niyrwp.yaml` check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Creating May 28,2025 11:36 UTC+0800 clusterdefinition.kubeblocks.io/name=mysql cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-226.us-west-2.compute.internal/172.31.13.226 May 28,2025 11:36 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-93.us-west-2.compute.internal/172.31.13.93 May 28,2025 11:38 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` connect checking... connect checking... connect checking... connect checking... connect checking... check cluster connect done `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check pod mysql-niyrwp-mysql-0 container_name mysql exist password 94kc9Lx1R7 check pod mysql-niyrwp-mysql-0 container_name mysql-exporter exist password 94kc9Lx1R7 check pod mysql-niyrwp-mysql-0 container_name kbagent exist password 94kc9Lx1R7 check pod mysql-niyrwp-mysql-0 container_name config-manager exist password 94kc9Lx1R7 No container logs contain secret password. describe cluster `kbcli cluster describe mysql-niyrwp --namespace ns-lweur ` Name: mysql-niyrwp Created Time: May 28,2025 11:36 UTC+0800 NAMESPACE CLUSTER-DEFINITION TOPOLOGY STATUS TERMINATION-POLICY ns-lweur mysql semisync-proxysql Running Delete Endpoints: COMPONENT INTERNAL EXTERNAL mysql mysql-niyrwp-mysql.ns-lweur.svc.cluster.local:3306 proxysql mysql-niyrwp-proxysql-proxy-ordinal-0.ns-lweur.svc.cluster.local:6032 mysql-niyrwp-proxysql-proxy-ordinal-0.ns-lweur.svc.cluster.local:6033 mysql-niyrwp-proxysql-proxy-server.ns-lweur.svc.cluster.local:6033 Topology: COMPONENT SERVICE-VERSION INSTANCE ROLE STATUS AZ NODE CREATED-TIME mysql 8.0.30 mysql-niyrwp-mysql-0 primary Running us-west-2a ip-172-31-13-226.us-west-2.compute.internal/172.31.13.226 May 28,2025 11:36 UTC+0800 mysql 8.0.30 mysql-niyrwp-mysql-1 secondary Running us-west-2a ip-172-31-13-93.us-west-2.compute.internal/172.31.13.93 May 28,2025 11:38 UTC+0800 proxysql 2.4.4 mysql-niyrwp-proxysql-0 Running us-west-2a ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 11:39 UTC+0800 Resources Allocation: COMPONENT INSTANCE-TEMPLATE CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS mysql 100m / 100m 512Mi / 512Mi data:10Gi kb-default-sc proxysql 100m / 100m 512Mi / 512Mi Images: COMPONENT COMPONENT-DEFINITION IMAGE mysql mysql-8.0-1.0.0-alpha.0 docker.io/apecloud/mysql:8.0.30 docker.io/apecloud/mysqld-exporter:0.15.1 docker.io/apecloud/kubeblocks-tools:1.0.0 proxysql proxysql-mysql-1.0.0-alpha.0 docker.io/apecloud/proxysql:2.4.4 Data Protection: BACKUP-REPO AUTO-BACKUP BACKUP-SCHEDULE BACKUP-METHOD BACKUP-RETENTION RECOVERABLE-TIME Show cluster events: kbcli cluster list-events -n ns-lweur mysql-niyrwp `kbcli cluster label mysql-niyrwp app.kubernetes.io/instance- --namespace ns-lweur ` label "app.kubernetes.io/instance" not found. `kbcli cluster label mysql-niyrwp app.kubernetes.io/instance=mysql-niyrwp --namespace ns-lweur ` `kbcli cluster label mysql-niyrwp --list --namespace ns-lweur ` NAME NAMESPACE LABELS mysql-niyrwp ns-lweur app.kubernetes.io/instance=mysql-niyrwp clusterdefinition.kubeblocks.io/name=mysql label cluster app.kubernetes.io/instance=mysql-niyrwp Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=mysql-niyrwp --namespace ns-lweur ` `kbcli cluster label mysql-niyrwp --list --namespace ns-lweur ` NAME NAMESPACE LABELS mysql-niyrwp ns-lweur app.kubernetes.io/instance=mysql-niyrwp case.name=kbcli.test1 clusterdefinition.kubeblocks.io/name=mysql label cluster case.name=kbcli.test1 Success `kbcli cluster label mysql-niyrwp case.name=kbcli.test2 --overwrite --namespace ns-lweur ` `kbcli cluster label mysql-niyrwp --list --namespace ns-lweur ` NAME NAMESPACE LABELS mysql-niyrwp ns-lweur app.kubernetes.io/instance=mysql-niyrwp case.name=kbcli.test2 clusterdefinition.kubeblocks.io/name=mysql label cluster case.name=kbcli.test2 Success `kbcli cluster label mysql-niyrwp case.name- --namespace ns-lweur ` `kbcli cluster label mysql-niyrwp --list --namespace ns-lweur ` NAME NAMESPACE LABELS mysql-niyrwp ns-lweur app.kubernetes.io/instance=mysql-niyrwp clusterdefinition.kubeblocks.io/name=mysql delete cluster label case.name Success cluster connect `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. *************************** 1. row *************************** Slave_IO_State: Waiting for source to send event Master_Host: mysql-niyrwp-mysql-0.mysql-niyrwp-mysql-headless Master_User: kbreplicator Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql-niyrwp-mysql-0-bin.000003 Read_Master_Log_Pos: 41184 Relay_Log_File: relay-bin.000002 Relay_Log_Pos: 14901 Relay_Master_Log_File: mysql-niyrwp-mysql-0-bin.000003 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 41184 Relay_Log_Space: 15105 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: Yes Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: 33dee2c2-3b75-11f0-b607-7ec94d11cac7 Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Replica has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: 33dee2c2-3b75-11f0-b607-7ec94d11cac7:74-109 Executed_Gtid_Set: 33dee2c2-3b75-11f0-b607-7ec94d11cac7:1-109 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: Master_public_key_path: Get_master_public_key: 0 Network_Namespace: `echo " echo \"show variables like 'max_connections';\" | mysql -h127.0.0.1 -uroot -p94kc9Lx1R7 -P3306 " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Variable_name Value max_connections 83 connect cluster Success set max_connections to 83 insert batch data by db client Error from server (NotFound): pods "test-db-client-executionloop-mysql-niyrwp" not found DB_CLIENT_BATCH_DATA_COUNT: `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-mysql-niyrwp --namespace ns-lweur ` Error from server (NotFound): pods "test-db-client-executionloop-mysql-niyrwp" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-executionloop-mysql-niyrwp" not found `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql apiVersion: v1 kind: Pod metadata: name: test-db-client-executionloop-mysql-niyrwp namespace: ns-lweur spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "mysql-niyrwp-mysql.ns-lweur.svc.cluster.local" - "--user" - "root" - "--password" - "94kc9Lx1R7" - "--port" - "3306" - "--dbtype" - "mysql" - "--test" - "executionloop" - "--duration" - "60" - "--interval" - "1" restartPolicy: Never `kubectl apply -f test-db-client-executionloop-mysql-niyrwp.yaml` pod/test-db-client-executionloop-mysql-niyrwp created apply test-db-client-executionloop-mysql-niyrwp.yaml Success `rm -rf test-db-client-executionloop-mysql-niyrwp.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-niyrwp 1/1 Running 0 6s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-niyrwp 1/1 Running 0 11s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-niyrwp 1/1 Running 0 17s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-niyrwp 1/1 Running 0 23s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-niyrwp 1/1 Running 0 28s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-niyrwp 1/1 Running 0 34s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-niyrwp 1/1 Running 0 40s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-niyrwp 1/1 Running 0 46s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-niyrwp 1/1 Running 0 52s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-niyrwp 1/1 Running 0 58s check pod test-db-client-executionloop-mysql-niyrwp status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mysql-niyrwp 0/1 Completed 0 64s check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-226.us-west-2.compute.internal/172.31.13.226 May 28,2025 11:36 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-93.us-west-2.compute.internal/172.31.13.93 May 28,2025 11:38 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done --host mysql-niyrwp-mysql.ns-lweur.svc.cluster.local --user root --password 94kc9Lx1R7 --port 3306 --dbtype mysql --test executionloop --duration 60 --interval 1 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider] 03:41:13.645 [main] DEBUG com.clickhouse.jdbc.ClickHouseDriver -- ClickHouse Driver 0.0.0.0(JDBC: 0.0.0.0) registered Execution loop start: create databases executions_loop CREATE DATABASE IF NOT EXISTS executions_loop; drop table executions_loop_table DROP TABLE IF EXISTS executions_loop.executions_loop_table; create table executions_loop_table CREATE TABLE IF NOT EXISTS executions_loop.executions_loop_table (id INT PRIMARY KEY AUTO_INCREMENT, value VARCHAR(255), tinyint_col TINYINT, smallint_col SMALLINT, mediumint_col MEDIUMINT, int_col INT, bigint_col BIGINT, float_col FLOAT, double_col DOUBLE, decimal_col DECIMAL(10, 2), date_col DATE, time_col TIME, datetime_col DATETIME, timestamp_col TIMESTAMP, year_col YEAR, char_col CHAR(10), text_col TEXT, blob_col BLOB, binary_col BINARY(10), varbinary_col VARBINARY(255), enum_col ENUM('Option1', 'Option2', 'Option3'), set_col SET('Value1', 'Value2', 'Value3') ); Execution loop start:INSERT INTO executions_loop.executions_loop_table (value, tinyint_col, smallint_col, mediumint_col, int_col, bigint_col, float_col, double_col, decimal_col, date_col, time_col, datetime_col, timestamp_col, year_col, char_col, text_col, blob_col, binary_col, varbinary_col, enum_col, set_col) VALUES ('executions_loop_test_1', 35, 2700, 6263627, -719219120, -1793661883340950983, 0.5021606, 0.7485395626040338, 44.3334982601318, '2025-05-28', '03:41:15', '2025-05-28 03:41:15.103', CURRENT_TIMESTAMP, 2054, 'CmE3ij9U2n', '0KpbCvRPoUFlSekQMnWKhNEiTTkOuJVYO3TgkeZxV4kefqe4B51Nh6oMTgXxhl6tpY2KadI89xFqg7grmm8SLCUblIGUgrW6ZNFQYiXwxNnpslPEezD73f6r083eplFHr3bip1YUcnncxJ9iEkBg2UGqVXQf9QD1caSgHvR5nbLKrPIjJikMO8VVFj09QHeFKZuxxWGfGLJNMol24RCDuZB5aKMpG8ma2N3Bp5ljEf2WqrpmQ9JBDgV8lFc77qe', UNHEX(REPLACE('16dc2e450232148ca884', ' ', '')), UNHEX(REPLACE('5cfda99d8c4211557c1e', ' ', '')), UNHEX(REPLACE('96b7762588db5efa3a4a4e91536e9aaceeaf2dca9532704a52ae8de7a3f02a00d3b16132c38f48b7e129b3c78993319aff6d604aeb395dd12014a741e56baac23e686d30e608fd550aee1b5c60f5cfca20d1d0aa2fcf2a952eb5a648b0ce5049e9c68d98c63982a6c60f89a81a766e02cc3187a5574ab05207a6e90338caac7c58af99f39bcff603ba3c2804fc52f23e3241a457fe7fc4b50b65c0e6f496755404b3e052829ae11fa0d50602cf00d465b5eb8761e2314bc7edc03a751f1973b1da9dcd4efa85d523c6bc252b601815a86b0b026f650f5038350f1af738159669b9d05f5268216c498d74c8a0005357d19feaf6500e465168beb26359827432', ' ', '')), 'Option2', 'Value1' ); [ 1s ] executions total: 9 successful: 9 failed: 0 disconnect: 0 [ 2s ] executions total: 28 successful: 28 failed: 0 disconnect: 0 [ 3s ] executions total: 46 successful: 46 failed: 0 disconnect: 0 [ 4s ] executions total: 63 successful: 63 failed: 0 disconnect: 0 [ 5s ] executions total: 80 successful: 80 failed: 0 disconnect: 0 [ 6s ] executions total: 95 successful: 95 failed: 0 disconnect: 0 [ 7s ] executions total: 114 successful: 114 failed: 0 disconnect: 0 [ 8s ] executions total: 132 successful: 132 failed: 0 disconnect: 0 [ 9s ] executions total: 148 successful: 148 failed: 0 disconnect: 0 [ 10s ] executions total: 165 successful: 165 failed: 0 disconnect: 0 [ 11s ] executions total: 180 successful: 180 failed: 0 disconnect: 0 [ 12s ] executions total: 198 successful: 198 failed: 0 disconnect: 0 [ 13s ] executions total: 222 successful: 222 failed: 0 disconnect: 0 [ 14s ] executions total: 242 successful: 242 failed: 0 disconnect: 0 [ 15s ] executions total: 263 successful: 263 failed: 0 disconnect: 0 [ 16s ] executions total: 283 successful: 283 failed: 0 disconnect: 0 [ 17s ] executions total: 306 successful: 306 failed: 0 disconnect: 0 [ 18s ] executions total: 325 successful: 325 failed: 0 disconnect: 0 [ 19s ] executions total: 343 successful: 343 failed: 0 disconnect: 0 [ 20s ] executions total: 364 successful: 364 failed: 0 disconnect: 0 [ 21s ] executions total: 385 successful: 385 failed: 0 disconnect: 0 [ 22s ] executions total: 402 successful: 402 failed: 0 disconnect: 0 [ 23s ] executions total: 420 successful: 420 failed: 0 disconnect: 0 [ 24s ] executions total: 440 successful: 440 failed: 0 disconnect: 0 [ 25s ] executions total: 458 successful: 458 failed: 0 disconnect: 0 [ 26s ] executions total: 479 successful: 479 failed: 0 disconnect: 0 [ 27s ] executions total: 502 successful: 502 failed: 0 disconnect: 0 [ 28s ] executions total: 523 successful: 523 failed: 0 disconnect: 0 [ 29s ] executions total: 542 successful: 542 failed: 0 disconnect: 0 [ 30s ] executions total: 562 successful: 562 failed: 0 disconnect: 0 [ 31s ] executions total: 583 successful: 583 failed: 0 disconnect: 0 [ 32s ] executions total: 603 successful: 603 failed: 0 disconnect: 0 [ 33s ] executions total: 624 successful: 624 failed: 0 disconnect: 0 [ 34s ] executions total: 645 successful: 645 failed: 0 disconnect: 0 [ 35s ] executions total: 665 successful: 665 failed: 0 disconnect: 0 [ 36s ] executions total: 688 successful: 688 failed: 0 disconnect: 0 [ 37s ] executions total: 706 successful: 706 failed: 0 disconnect: 0 [ 38s ] executions total: 724 successful: 724 failed: 0 disconnect: 0 [ 39s ] executions total: 743 successful: 743 failed: 0 disconnect: 0 [ 40s ] executions total: 762 successful: 762 failed: 0 disconnect: 0 [ 41s ] executions total: 783 successful: 783 failed: 0 disconnect: 0 [ 42s ] executions total: 800 successful: 800 failed: 0 disconnect: 0 [ 43s ] executions total: 819 successful: 819 failed: 0 disconnect: 0 [ 44s ] executions total: 837 successful: 837 failed: 0 disconnect: 0 [ 45s ] executions total: 858 successful: 858 failed: 0 disconnect: 0 [ 46s ] executions total: 877 successful: 877 failed: 0 disconnect: 0 [ 47s ] executions total: 890 successful: 890 failed: 0 disconnect: 0 [ 48s ] executions total: 913 successful: 913 failed: 0 disconnect: 0 [ 49s ] executions total: 935 successful: 935 failed: 0 disconnect: 0 [ 50s ] executions total: 954 successful: 954 failed: 0 disconnect: 0 [ 51s ] executions total: 967 successful: 967 failed: 0 disconnect: 0 [ 52s ] executions total: 984 successful: 984 failed: 0 disconnect: 0 [ 53s ] executions total: 998 successful: 998 failed: 0 disconnect: 0 [ 54s ] executions total: 1013 successful: 1013 failed: 0 disconnect: 0 [ 55s ] executions total: 1032 successful: 1032 failed: 0 disconnect: 0 [ 56s ] executions total: 1045 successful: 1045 failed: 0 disconnect: 0 [ 57s ] executions total: 1063 successful: 1063 failed: 0 disconnect: 0 [ 58s ] executions total: 1081 successful: 1081 failed: 0 disconnect: 0 [ 60s ] executions total: 1084 successful: 1084 failed: 0 disconnect: 0 Test Result: Total Executions: 1084 Successful Executions: 1084 Failed Executions: 0 Disconnection Counts: 0 Connection Information: Database Type: mysql Host: mysql-niyrwp-mysql.ns-lweur.svc.cluster.local Port: 3306 Database: Table: User: root Org: Access Mode: mysql Test Type: executionloop Query: Duration: 60 seconds Interval: 1 seconds DB_CLIENT_BATCH_DATA_COUNT: 1084 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-mysql-niyrwp --namespace ns-lweur ` pod/test-db-client-executionloop-mysql-niyrwp patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-executionloop-mysql-niyrwp" force deleted LB_TYPE is set to: internet cluster expose check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster expose mysql-niyrwp --auto-approve --force=true --type internet --enable true --components mysql --role-selector primary --namespace ns-lweur ` OpsRequest mysql-niyrwp-expose-l4ndw created successfully, you can view the progress: kbcli cluster describe-ops mysql-niyrwp-expose-l4ndw -n ns-lweur check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-expose-l4ndw ns-lweur Expose mysql-niyrwp mysql Running 0/1 May 28,2025 11:42 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-226.us-west-2.compute.internal/172.31.13.226 May 28,2025 11:36 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-93.us-west-2.compute.internal/172.31.13.93 May 28,2025 11:38 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-expose-l4ndw ns-lweur Expose mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:42 UTC+0800 check ops status done ops_status:mysql-niyrwp-expose-l4ndw ns-lweur Expose mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:42 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-expose-l4ndw --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-expose-l4ndw patched `kbcli cluster delete-ops --name mysql-niyrwp-expose-l4ndw --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-expose-l4ndw deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success cluster configure component_tmp: mysql apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-reconfiguring- namespace: ns-lweur spec: type: Reconfiguring clusterName: mysql-niyrwp force: true reconfigures: - componentName: mysql parameters: - key: binlog_expire_logs_seconds value: '691200' check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-reconfiguring-dcn6g created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-reconfiguring-dcn6g ns-lweur Reconfiguring mysql-niyrwp mysql,mysql Running -/- May 28,2025 11:43 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-226.us-west-2.compute.internal/172.31.13.226 May 28,2025 11:36 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-93.us-west-2.compute.internal/172.31.13.93 May 28,2025 11:38 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 11:39 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-reconfiguring-dcn6g ns-lweur Reconfiguring mysql-niyrwp mysql,mysql Succeed -/- May 28,2025 11:43 UTC+0800 check ops status done ops_status:mysql-niyrwp-reconfiguring-dcn6g ns-lweur Reconfiguring mysql-niyrwp mysql,mysql Succeed -/- May 28,2025 11:43 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-reconfiguring-dcn6g --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-reconfiguring-dcn6g patched `kbcli cluster delete-ops --name mysql-niyrwp-reconfiguring-dcn6g --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-reconfiguring-dcn6g deleted component_config:mysql check config variables `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql mysql mysql Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. echo "echo \"show variables like 'binlog_expire_logs_seconds'\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash | (grep "binlog_expire_logs_seconds" || true) | awk '***print mysql***' current value_actual: 691200 configure:[binlog_expire_logs_seconds] result actual:[691200] equal expected:[691200] `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart mysql-niyrwp --auto-approve --force=true --namespace ns-lweur ` OpsRequest mysql-niyrwp-restart-9r6mg created successfully, you can view the progress: kbcli cluster describe-ops mysql-niyrwp-restart-9r6mg -n ns-lweur check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-restart-9r6mg ns-lweur Restart mysql-niyrwp mysql,proxysql Running 0/3 May 28,2025 11:44 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:46 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:44 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-5-153.us-west-2.compute.internal/172.31.5.153 May 28,2025 11:48 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-restart-9r6mg ns-lweur Restart mysql-niyrwp mysql,proxysql Succeed 3/3 May 28,2025 11:44 UTC+0800 check ops status done ops_status:mysql-niyrwp-restart-9r6mg ns-lweur Restart mysql-niyrwp mysql,proxysql Succeed 3/3 May 28,2025 11:44 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-restart-9r6mg --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-restart-9r6mg patched `kbcli cluster delete-ops --name mysql-niyrwp-restart-9r6mg --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-restart-9r6mg deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success cluster stop check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster stop mysql-niyrwp --auto-approve --force=true --namespace ns-lweur ` OpsRequest mysql-niyrwp-stop-j5wk5 created successfully, you can view the progress: kbcli cluster describe-ops mysql-niyrwp-stop-j5wk5 -n ns-lweur check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-stop-j5wk5 ns-lweur Stop mysql-niyrwp mysql,proxysql Running 0/3 May 28,2025 11:49 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Stopping May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping check cluster status done cluster_status:Stopped check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME check pod status done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-stop-j5wk5 ns-lweur Stop mysql-niyrwp mysql,proxysql Succeed 3/3 May 28,2025 11:49 UTC+0800 check ops status done ops_status:mysql-niyrwp-stop-j5wk5 ns-lweur Stop mysql-niyrwp mysql,proxysql Succeed 3/3 May 28,2025 11:49 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-stop-j5wk5 --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-stop-j5wk5 patched `kbcli cluster delete-ops --name mysql-niyrwp-stop-j5wk5 --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-stop-j5wk5 deleted cluster start check cluster status before ops check cluster status done cluster_status:Stopped `kbcli cluster start mysql-niyrwp --force=true --namespace ns-lweur ` OpsRequest mysql-niyrwp-start-hbxkn created successfully, you can view the progress: kbcli cluster describe-ops mysql-niyrwp-start-hbxkn -n ns-lweur check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-start-hbxkn ns-lweur Start mysql-niyrwp mysql,proxysql Running 0/3 May 28,2025 11:50 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:50 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-start-hbxkn ns-lweur Start mysql-niyrwp mysql,proxysql Succeed 3/3 May 28,2025 11:50 UTC+0800 check ops status done ops_status:mysql-niyrwp-start-hbxkn ns-lweur Start mysql-niyrwp mysql,proxysql Succeed 3/3 May 28,2025 11:50 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-start-hbxkn --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-start-hbxkn patched `kbcli cluster delete-ops --name mysql-niyrwp-start-hbxkn --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-start-hbxkn deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover dnsrandom check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mysql-niyrwp" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnsrandom-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-0 mode: all action: random duration: 2m `kubectl apply -f test-chaos-mesh-dnsrandom-mysql-niyrwp.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnsrandom-mysql-niyrwp created apply test-chaos-mesh-dnsrandom-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-dnsrandom-mysql-niyrwp.yaml` dnsrandom chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:50 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mysql-niyrwp" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mysql-niyrwp" not found check failover pod name failover pod name:mysql-niyrwp-mysql-0 failover dnsrandom Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test switchover apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-switchover- namespace: ns-lweur spec: clusterName: mysql-niyrwp type: Switchover switchover: - componentName: mysql candidateName: 'mysql-niyrwp-mysql-1' instanceName: 'mysql-niyrwp-mysql-0' `kubectl create -f test_create_mysql-niyrwp_switchover.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-switchover-ndw4v created create test_create_mysql-niyrwp_switchover.yaml Success `rm -rf test_create_mysql-niyrwp_switchover.yaml` get mysql primary pod name... switchover pod:mysql-niyrwp-mysql-1 switchover success No resources found in ns-lweur namespace. `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql apiVersion: v1 kind: Pod metadata: name: benchtest-mysql-niyrwp namespace: ns-lweur spec: containers: - name: test-sysbench imagePullPolicy: IfNotPresent image: docker.io/apecloud/customsuites:latest env: - name: TYPE value: "2" - name: FLAG value: "0" - name: CONFIGS value: "mode:all,driver:mysql,host:mysql-niyrwp-mysql.ns-lweur.svc.cluster.local,user:root,password:94kc9Lx1R7,port:3306,db:benchtest,tables:5,threads:4,times:10,size:1000,type:oltp_read_write" restartPolicy: Never `kubectl apply -f benchtest-mysql-niyrwp.yaml` pod/benchtest-mysql-niyrwp created apply benchtest-mysql-niyrwp.yaml Success `rm -rf benchtest-mysql-niyrwp.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-niyrwp 0/1 ContainerCreating 0 0s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-niyrwp 1/1 Running 0 5s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-niyrwp 1/1 Running 0 11s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-niyrwp 1/1 Running 0 17s check pod benchtest-mysql-niyrwp status done pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-niyrwp 0/1 Completed 0 23s `kubectl logs benchtest-mysql-niyrwp --tail 30 --namespace ns-lweur ` [ 7s ] thds: 4 tps: 12.00 qps: 211.96 (r/w/o: 146.97/41.99/23.00) lat (ms,99%): 502.20 err/s: 0.00 reconn/s: 0.00 [ 8s ] thds: 4 tps: 8.00 qps: 169.01 (r/w/o: 116.01/36.00/17.00) lat (ms,99%): 694.45 err/s: 0.00 reconn/s: 0.00 [ 9s ] thds: 4 tps: 9.00 qps: 196.02 (r/w/o: 137.01/41.00/18.00) lat (ms,99%): 694.45 err/s: 0.00 reconn/s: 0.00 [ 10s ] thds: 4 tps: 11.00 qps: 216.00 (r/w/o: 153.00/41.00/22.00) lat (ms,99%): 502.20 err/s: 0.00 reconn/s: 0.00 SQL statistics: queries performed: read: 1330 write: 380 other: 190 total: 1900 transactions: 95 (9.31 per sec.) queries: 1900 (186.22 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 10.2012s total number of events: 95 Latency (ms): min: 200.03 avg: 429.36 max: 701.95 99th percentile: 694.45 sum: 40789.20 Threads fairness: events (avg/stddev): 23.7500/0.83 execution time (avg/stddev): 10.1973/0.00 `kubectl delete pod benchtest-mysql-niyrwp --force --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "benchtest-mysql-niyrwp" force deleted LB_TYPE is set to: internet No resources found in ns-lweur namespace. `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql apiVersion: v1 kind: Pod metadata: name: benchtest-mysql-niyrwp namespace: ns-lweur spec: containers: - name: test-sysbench imagePullPolicy: IfNotPresent image: docker.io/apecloud/customsuites:latest env: - name: TYPE value: "2" - name: FLAG value: "0" - name: CONFIGS value: "mode:all,driver:mysql,host:a730c4cdac96644e0b18d820582ff119-617d4070bcd9aa23.elb.us-west-2.amazonaws.com,user:root,password:94kc9Lx1R7,port:3306,db:benchtest,tables:5,threads:4,times:10,size:1000,type:oltp_read_write" restartPolicy: Never `kubectl apply -f benchtest-mysql-niyrwp.yaml` pod/benchtest-mysql-niyrwp created apply benchtest-mysql-niyrwp.yaml Success `rm -rf benchtest-mysql-niyrwp.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-niyrwp 0/1 ContainerCreating 0 1s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-niyrwp 1/1 Running 0 6s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-niyrwp 1/1 Running 0 12s pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-niyrwp 1/1 Running 0 18s check pod benchtest-mysql-niyrwp status done pod_status:NAME READY STATUS RESTARTS AGE benchtest-mysql-niyrwp 0/1 Completed 0 24s `kubectl logs benchtest-mysql-niyrwp --tail 30 --namespace ns-lweur ` [ 7s ] thds: 4 tps: 8.00 qps: 163.00 (r/w/o: 115.00/32.00/16.00) lat (ms,99%): 601.29 err/s: 0.00 reconn/s: 0.00 [ 8s ] thds: 4 tps: 9.97 qps: 165.47 (r/w/o: 111.64/35.88/17.94) lat (ms,99%): 601.29 err/s: 0.00 reconn/s: 0.00 [ 9s ] thds: 4 tps: 7.02 qps: 180.59 (r/w/o: 126.41/38.12/16.05) lat (ms,99%): 694.45 err/s: 0.00 reconn/s: 0.00 [ 10s ] thds: 4 tps: 9.98 qps: 174.66 (r/w/o: 120.76/33.93/19.96) lat (ms,99%): 694.45 err/s: 0.00 reconn/s: 0.00 SQL statistics: queries performed: read: 1218 write: 348 other: 174 total: 1740 transactions: 87 (8.37 per sec.) queries: 1740 (167.35 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 10.3955s total number of events: 87 Latency (ms): min: 205.47 avg: 470.02 max: 892.25 99th percentile: 787.74 sum: 40892.11 Threads fairness: events (avg/stddev): 21.7500/1.09 execution time (avg/stddev): 10.2230/0.13 `kubectl delete pod benchtest-mysql-niyrwp --force --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "benchtest-mysql-niyrwp" force deleted test failover networkcorruptover check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-niyrwp" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorruptover-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-1 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorruptover-mysql-niyrwp.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-mysql-niyrwp created apply test-chaos-mesh-networkcorruptover-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-networkcorruptover-mysql-niyrwp.yaml` networkcorruptover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:50 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-niyrwp" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mysql-niyrwp" not found check failover pod name failover pod name:mysql-niyrwp-mysql-0 failover networkcorruptover Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover networklossover check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-niyrwp" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-0 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-mysql-niyrwp.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-mysql-niyrwp created apply test-chaos-mesh-networklossover-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-networklossover-mysql-niyrwp.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:50 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-0 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-niyrwp" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mysql-niyrwp" not found check failover pod name failover pod name:mysql-niyrwp-mysql-1 failover networklossover Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check readonly db_client batch data Success cluster does not need to check monitor currently check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:50 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-0 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check cluster connect done test failover podfailure check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mysql-niyrwp" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podfailure-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-1 mode: all action: pod-failure duration: 2m `kubectl apply -f test-chaos-mesh-podfailure-mysql-niyrwp.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podfailure-mysql-niyrwp created apply test-chaos-mesh-podfailure-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-podfailure-mysql-niyrwp.yaml` podfailure chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:50 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mysql-niyrwp" force deleted Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mysql-niyrwp" not found check failover pod name failover pod name:mysql-niyrwp-mysql-0 failover podfailure Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover fullcpu check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mysql-niyrwp" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-fullcpu-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-0 mode: all stressors: cpu: workers: 100 load: 100 duration: 2m `kubectl apply -f test-chaos-mesh-fullcpu-mysql-niyrwp.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-fullcpu-mysql-niyrwp created apply test-chaos-mesh-fullcpu-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-fullcpu-mysql-niyrwp.yaml` fullcpu chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:50 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:10Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mysql-niyrwp" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mysql-niyrwp" not found check failover pod name failover pod name:mysql-niyrwp-mysql-0 failover fullcpu Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success `kubectl get pvc -l app.kubernetes.io/instance=mysql-niyrwp,apps.kubeblocks.io/component-name=mysql,apps.kubeblocks.io/vct-name=data --namespace ns-lweur ` cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in mysql-niyrwp namespace. `kbcli cluster volume-expand mysql-niyrwp --auto-approve --force=true --components mysql --volume-claim-templates data --storage 15Gi --namespace ns-lweur ` OpsRequest mysql-niyrwp-volumeexpansion-2bhfb created successfully, you can view the progress: kbcli cluster describe-ops mysql-niyrwp-volumeexpansion-2bhfb -n ns-lweur check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-volumeexpansion-2bhfb ns-lweur VolumeExpansion mysql-niyrwp mysql Running 0/2 May 28,2025 12:09 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:15Gi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:50 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:15Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done No resources found in mysql-niyrwp namespace. check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-volumeexpansion-2bhfb ns-lweur VolumeExpansion mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:09 UTC+0800 check ops status done ops_status:mysql-niyrwp-volumeexpansion-2bhfb ns-lweur VolumeExpansion mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:09 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-volumeexpansion-2bhfb --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-volumeexpansion-2bhfb patched `kbcli cluster delete-ops --name mysql-niyrwp-volumeexpansion-2bhfb --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-volumeexpansion-2bhfb deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover delete pod:mysql-niyrwp-mysql-0 `kubectl delete pod mysql-niyrwp-mysql-0 --namespace ns-lweur ` pod "mysql-niyrwp-mysql-0" deleted check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:15Gi ip-172-31-13-226.us-west-2.compute.internal/172.31.13.226 May 28,2025 12:12 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:15Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-0 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check cluster connect done check failover pod name failover pod name:mysql-niyrwp-mysql-1 failover Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover networkbandwidthover check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mysql-niyrwp" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkbandwidthover-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-1 action: bandwidth mode: all bandwidth: rate: '1bps' limit: 20971520 buffer: 10000 duration: 2m `kubectl apply -f test-chaos-mesh-networkbandwidthover-mysql-niyrwp.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkbandwidthover-mysql-niyrwp created apply test-chaos-mesh-networkbandwidthover-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-networkbandwidthover-mysql-niyrwp.yaml` networkbandwidthover chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:15Gi ip-172-31-13-226.us-west-2.compute.internal/172.31.13.226 May 28,2025 12:12 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:15Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mysql-niyrwp" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mysql-niyrwp" not found check failover pod name failover pod name:mysql-niyrwp-mysql-0 failover networkbandwidthover Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover drainnode check node drain check node drain success kubectl get pod mysql-niyrwp-mysql-0 --namespace ns-lweur -o jsonpath='***.spec.nodeName***' get node name:ip-172-31-13-226.us-west-2.compute.internal success check if multiple pods are on the same node kubectl get pod mysql-niyrwp-mysql-1 --namespace ns-lweur -o jsonpath='***.spec.nodeName***' get node name:ip-172-31-9-182.us-west-2.compute.internal success kubectl drain ip-172-31-13-226.us-west-2.compute.internal --delete-emptydir-data --ignore-daemonsets --force --grace-period 0 --timeout 60s node/ip-172-31-13-226.us-west-2.compute.internal cordoned Warning: ignoring DaemonSet-managed Pods: chaos-mesh/chaos-daemon-tp4jn, kube-system/aws-node-mrmqt, kube-system/ebs-csi-node-hx55f, kube-system/kube-proxy-gcjms evicting pod ns-sxczu/apemysql-hmdmte-mysql-2 evicting pod ns-lweur/mysql-niyrwp-mysql-0 evicting pod ns-cbpst/mysql-fphydk-mysql-0 evicting pod ns-pkuvx/postgres-cyetms-postgresql-0 pod/postgres-cyetms-postgresql-0 evicted pod/mysql-fphydk-mysql-0 evicted pod/apemysql-hmdmte-mysql-2 evicted pod/mysql-niyrwp-mysql-0 evicted node/ip-172-31-13-226.us-west-2.compute.internal drained kubectl uncordon ip-172-31-13-226.us-west-2.compute.internal node/ip-172-31-13-226.us-west-2.compute.internal uncordoned check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:15Gi ip-172-31-13-93.us-west-2.compute.internal/172.31.13.93 May 28,2025 12:16 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:15Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-0 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check cluster connect done check failover pod name failover pod name:mysql-niyrwp-mysql-1 failover drainnode Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover kill1 check node drain check node drain success `kill 1` Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file exec return message: check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:15Gi ip-172-31-13-93.us-west-2.compute.internal/172.31.13.93 May 28,2025 12:16 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:15Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check failover pod name failover pod name:mysql-niyrwp-mysql-0 failover kill1 Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover networkdelay check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mysql-niyrwp" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkdelay-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-0 mode: all action: delay delay: latency: 2000ms correlation: '100' jitter: 0ms direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkdelay-mysql-niyrwp.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkdelay-mysql-niyrwp created apply test-chaos-mesh-networkdelay-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-networkdelay-mysql-niyrwp.yaml` networkdelay chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 100m / 100m 512Mi / 512Mi data:15Gi ip-172-31-13-93.us-west-2.compute.internal/172.31.13.93 May 28,2025 12:16 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 100m / 100m 512Mi / 512Mi data:15Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 11:50 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mysql-niyrwp" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mysql-niyrwp" not found check failover pod name failover pod name:mysql-niyrwp-mysql-0 failover networkdelay Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale mysql-niyrwp --auto-approve --force=true --components mysql --cpu 200m --memory 0.6Gi --namespace ns-lweur ` OpsRequest mysql-niyrwp-verticalscaling-hszmm created successfully, you can view the progress: kbcli cluster describe-ops mysql-niyrwp-verticalscaling-hszmm -n ns-lweur check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-verticalscaling-hszmm ns-lweur VerticalScaling mysql-niyrwp mysql Running 0/2 May 28,2025 12:22 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-128.us-west-2.compute.internal/172.31.13.128 May 28,2025 11:51 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-verticalscaling-hszmm ns-lweur VerticalScaling mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:22 UTC+0800 check ops status done ops_status:mysql-niyrwp-verticalscaling-hszmm ns-lweur VerticalScaling mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:22 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-verticalscaling-hszmm --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-verticalscaling-hszmm patched `kbcli cluster delete-ops --name mysql-niyrwp-verticalscaling-hszmm --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-verticalscaling-hszmm deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success cluster mysql scale-out check cluster status before ops check cluster status done cluster_status:Running No resources found in mysql-niyrwp namespace. `kbcli cluster scale-out mysql-niyrwp --auto-approve --force=true --components mysql --replicas 1 --namespace ns-lweur ` OpsRequest mysql-niyrwp-horizontalscaling-6dc5z created successfully, you can view the progress: kbcli cluster describe-ops mysql-niyrwp-horizontalscaling-6dc5z -n ns-lweur check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-horizontalscaling-6dc5z ns-lweur HorizontalScaling mysql-niyrwp mysql Running 0/1 May 28,2025 12:25 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-mysql-2 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 12:25 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:25 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 mysql-niyrwp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done No resources found in mysql-niyrwp namespace. check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-horizontalscaling-6dc5z ns-lweur HorizontalScaling mysql-niyrwp mysql Succeed 1/1 May 28,2025 12:25 UTC+0800 check ops status done ops_status:mysql-niyrwp-horizontalscaling-6dc5z ns-lweur HorizontalScaling mysql-niyrwp mysql Succeed 1/1 May 28,2025 12:25 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-horizontalscaling-6dc5z --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-horizontalscaling-6dc5z patched `kbcli cluster delete-ops --name mysql-niyrwp-horizontalscaling-6dc5z --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-horizontalscaling-6dc5z deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success cluster mysql scale-in check cluster status before ops check cluster status done cluster_status:Running No resources found in mysql-niyrwp namespace. `kbcli cluster scale-in mysql-niyrwp --auto-approve --force=true --components mysql --replicas 1 --namespace ns-lweur ` OpsRequest mysql-niyrwp-horizontalscaling-l7z7j created successfully, you can view the progress: kbcli cluster describe-ops mysql-niyrwp-horizontalscaling-l7z7j -n ns-lweur check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-horizontalscaling-l7z7j ns-lweur HorizontalScaling mysql-niyrwp mysql Running 0/1 May 28,2025 12:26 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done No resources found in mysql-niyrwp namespace. check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-horizontalscaling-l7z7j ns-lweur HorizontalScaling mysql-niyrwp mysql Succeed 1/1 May 28,2025 12:26 UTC+0800 check ops status done ops_status:mysql-niyrwp-horizontalscaling-l7z7j ns-lweur HorizontalScaling mysql-niyrwp mysql Succeed 1/1 May 28,2025 12:26 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-horizontalscaling-l7z7j --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-horizontalscaling-l7z7j patched `kbcli cluster delete-ops --name mysql-niyrwp-horizontalscaling-l7z7j --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-horizontalscaling-l7z7j deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover networkpartition check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-mysql-niyrwp" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkpartition-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-0 action: partition mode: all target: mode: all selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-1 direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkpartition-mysql-niyrwp.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkpartition-mysql-niyrwp created apply test-chaos-mesh-networkpartition-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-networkpartition-mysql-niyrwp.yaml` networkpartition chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-mysql-niyrwp" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networkpartition-mysql-niyrwp patched check failover pod name failover pod name:mysql-niyrwp-mysql-0 failover networkpartition Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover dnserror check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mysql-niyrwp" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnserror-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-0 mode: all action: error duration: 2m `kubectl apply -f test-chaos-mesh-dnserror-mysql-niyrwp.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnserror-mysql-niyrwp created apply test-chaos-mesh-dnserror-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-dnserror-mysql-niyrwp.yaml` dnserror chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mysql-niyrwp" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mysql-niyrwp" not found check failover pod name failover pod name:mysql-niyrwp-mysql-0 failover dnserror Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover networkduplicate check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mysql-niyrwp" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkduplicate-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-0 mode: all action: duplicate duplicate: duplicate: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkduplicate-mysql-niyrwp.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkduplicate-mysql-niyrwp created apply test-chaos-mesh-networkduplicate-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-networkduplicate-mysql-niyrwp.yaml` networkduplicate chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mysql-niyrwp" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mysql-niyrwp" not found check failover pod name failover pod name:mysql-niyrwp-mysql-0 failover networkduplicate Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover timeoffset check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mysql-niyrwp" not found Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: TimeChaos metadata: name: test-chaos-mesh-timeoffset-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-0 mode: all timeOffset: '-10m' clockIds: - CLOCK_REALTIME duration: 2m `kubectl apply -f test-chaos-mesh-timeoffset-mysql-niyrwp.yaml` timechaos.chaos-mesh.org/test-chaos-mesh-timeoffset-mysql-niyrwp created apply test-chaos-mesh-timeoffset-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-timeoffset-mysql-niyrwp.yaml` timeoffset chaos test waiting 120 seconds check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mysql-niyrwp" force deleted Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mysql-niyrwp" not found check failover pod name failover pod name:mysql-niyrwp-mysql-0 failover timeoffset Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover oom check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-0 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-mysql-niyrwp.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mysql-niyrwp created apply test-chaos-mesh-oom-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-oom-mysql-niyrwp.yaml` check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" force deleted stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mysql-niyrwp patched check failover pod name failover pod name:mysql-niyrwp-mysql-0 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-0 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-mysql-niyrwp.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mysql-niyrwp created apply test-chaos-mesh-oom-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-oom-mysql-niyrwp.yaml` check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Abnormal May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found failover pod name:mysql-niyrwp-mysql-0 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-0 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-mysql-niyrwp.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mysql-niyrwp created apply test-chaos-mesh-oom-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-oom-mysql-niyrwp.yaml` check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Abnormal May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Abnormal cluster_status:Abnormal check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" force deleted stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mysql-niyrwp patched failover pod name:mysql-niyrwp-mysql-0 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-mysql-niyrwp namespace: ns-lweur spec: selector: namespaces: - ns-lweur labelSelectors: apps.kubeblocks.io/pod-name: mysql-niyrwp-mysql-0 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-mysql-niyrwp.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mysql-niyrwp created apply test-chaos-mesh-oom-mysql-niyrwp.yaml Success `rm -rf test-chaos-mesh-oom-mysql-niyrwp.yaml` check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Abnormal May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Abnormal cluster_status:Abnormal cluster_status:Abnormal cluster_status:Abnormal cluster_status:Abnormal check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-0 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check cluster connect done `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mysql-niyrwp --namespace ns-lweur ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mysql-niyrwp" not found check failover pod name:mysql-niyrwp-mysql-1 failover oom Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check readonly db_client batch data Success test failover connectionstress check node drain check node drain success Error from server (NotFound): pods "test-db-client-connectionstress-mysql-niyrwp" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstress-mysql-niyrwp --namespace ns-lweur ` Error from server (NotFound): pods "test-db-client-connectionstress-mysql-niyrwp" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-connectionstress-mysql-niyrwp" not found `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql apiVersion: v1 kind: Pod metadata: name: test-db-client-connectionstress-mysql-niyrwp namespace: ns-lweur spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "mysql-niyrwp-mysql.ns-lweur.svc.cluster.local" - "--user" - "root" - "--password" - "94kc9Lx1R7" - "--port" - "3306" - "--database" - "mysql" - "--dbtype" - "mysql" - "--test" - "connectionstress" - "--connections" - "83" - "--duration" - "60" restartPolicy: Never `kubectl apply -f test-db-client-connectionstress-mysql-niyrwp.yaml` pod/test-db-client-connectionstress-mysql-niyrwp created apply test-db-client-connectionstress-mysql-niyrwp.yaml Success `rm -rf test-db-client-connectionstress-mysql-niyrwp.yaml` check pod status check pod test-db-client-connectionstress-mysql-niyrwp status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mysql-niyrwp 0/1 Completed 0 6s check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-0 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check cluster connect done --host mysql-niyrwp-mysql.ns-lweur.svc.cluster.local --user root --password 94kc9Lx1R7 --port 3306 --database mysql --dbtype mysql --test connectionstress --connections 83 --duration 60 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider] 04:42:22.720 [main] DEBUG com.clickhouse.jdbc.ClickHouseDriver -- ClickHouse Driver 0.0.0.0(JDBC: 0.0.0.0) registered Test Result: null Connection Information: Database Type: mysql Host: mysql-niyrwp-mysql.ns-lweur.svc.cluster.local Port: 3306 Database: mysql Table: User: root Org: Access Mode: mysql Test Type: connectionstress Connection Count: 83 Duration: 60 seconds `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstress-mysql-niyrwp --namespace ns-lweur ` pod/test-db-client-connectionstress-mysql-niyrwp patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-connectionstress-mysql-niyrwp" force deleted check failover pod name failover pod name:mysql-niyrwp-mysql-1 failover connectionstress Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check readonly db_client batch data Success cmpv upgrade service version:2,5.7.44|2,8.0.30|2,8.0.31|2,8.0.32|2,8.0.33|2,8.0.34|2,8.0.35|2,8.0.36|2,8.0.37|2,8.0.38|2,8.0.39|2,8.4.0|2,8.4.1|2,8.4.2 set latest cmpv service version latest service version:8.0.39 cmpv service version upgrade and downgrade 8.0.30 is less than 8.0.31 upgrade from:8.0.30 to service version:8.0.31 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.31 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-q2754 created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-q2754 ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 12:43 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-0 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-q2754 ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:43 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-q2754 ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:43 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-q2754 --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-q2754 patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-q2754 --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-q2754 deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check readonly db_client batch data Success 8.0.31 is less than 8.0.32 upgrade from:8.0.31 to service version:8.0.32 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.32 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-bpxjm created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-bpxjm ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 12:45 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-0 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-bpxjm ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:45 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-bpxjm ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:45 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-bpxjm --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-bpxjm patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-bpxjm --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-bpxjm deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check readonly db_client batch data Success 8.0.32 is less than 8.0.33 upgrade from:8.0.32 to service version:8.0.33 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.33 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-n8r42 created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-n8r42 ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 12:47 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-0 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-n8r42 ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:47 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-n8r42 ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:47 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-n8r42 --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-n8r42 patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-n8r42 --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-n8r42 deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check readonly db_client batch data Success 8.0.33 is less than 8.0.34 upgrade from:8.0.33 to service version:8.0.34 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.34 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-6c4gw created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-6c4gw ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 12:50 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-0 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-6c4gw ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:50 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-6c4gw ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:50 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-6c4gw --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-6c4gw patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-6c4gw --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-6c4gw deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check readonly db_client batch data Success upgrade from:8.0.34 to service version:8.0.35 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.35 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-xftsf created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-xftsf ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 12:52 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-0 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-xftsf ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:52 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-xftsf ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:52 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-xftsf --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-xftsf patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-xftsf --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-xftsf deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check readonly db_client batch data Success upgrade from:8.0.35 to service version:8.0.39 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.39 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-br6x2 created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-br6x2 ns-lweur Upgrade mysql-niyrwp Running -/- May 28,2025 12:54 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-0 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-br6x2 ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:54 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-br6x2 ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:54 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-br6x2 --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-br6x2 patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-br6x2 --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-br6x2 deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check readonly db_client batch data Success downgrade from:8.0.39 to service version:8.0.35 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.35 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-5ncrf created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-5ncrf ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 12:56 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-1;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-5ncrf ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:56 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-5ncrf ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:56 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-5ncrf --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-5ncrf patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-5ncrf --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-5ncrf deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success upgrade from:8.0.35 to service version:8.0.36 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.36 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-xvnb8 created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-xvnb8 ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 12:57 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-xvnb8 ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:57 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-xvnb8 ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:57 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-xvnb8 --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-xvnb8 patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-xvnb8 --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-xvnb8 deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success upgrade from:8.0.36 to service version:8.0.39 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.39 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-b646w created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-b646w ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 12:59 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-b646w ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:59 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-b646w ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 12:59 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-b646w --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-b646w patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-b646w --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-b646w deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success downgrade from:8.0.39 to service version:8.0.36 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.36 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-dpnnj created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-dpnnj ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 13:00 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-dpnnj ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:00 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-dpnnj ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:00 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-dpnnj --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-dpnnj patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-dpnnj --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-dpnnj deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success upgrade from:8.0.36 to service version:8.0.37 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.37 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-4dpft created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-4dpft ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 13:02 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-4dpft ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:02 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-4dpft ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:02 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-4dpft --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-4dpft patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-4dpft --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-4dpft deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success upgrade from:8.0.37 to service version:8.0.39 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.39 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-475dc created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-475dc ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 13:03 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-475dc ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:03 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-475dc ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:03 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-475dc --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-475dc patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-475dc --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-475dc deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success downgrade from:8.0.39 to service version:8.0.37 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.37 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-dvv5q created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-dvv5q ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 13:05 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-dvv5q ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:05 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-dvv5q ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:05 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-dvv5q --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-dvv5q patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-dvv5q --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-dvv5q deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success upgrade from:8.0.37 to service version:8.0.38 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.38 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-rrpgx created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-rrpgx ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 13:06 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-rrpgx ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:06 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-rrpgx ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:06 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-rrpgx --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-rrpgx patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-rrpgx --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-rrpgx deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success upgrade from:8.0.38 to service version:8.0.39 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.39 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-9gz52 created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-9gz52 ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 13:08 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 No resources found in ns-lweur namespace. primary: ;secondary: mysql-niyrwp-mysql-0 mysql-niyrwp-mysql-1 check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-9gz52 ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:08 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-9gz52 ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:08 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-9gz52 --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-9gz52 patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-9gz52 --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-9gz52 deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success downgrade from:8.0.39 to service version:8.0.38 cluster upgrade apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-upgrade-cmpv- namespace: ns-lweur spec: clusterName: mysql-niyrwp upgrade: components: - componentName: mysql serviceVersion: 8.0.38 type: Upgrade check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-rzd2l created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-rzd2l ns-lweur Upgrade mysql-niyrwp mysql Running 0/2 May 28,2025 13:09 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql Delete Updating May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-upgrade-cmpv-rzd2l ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:09 UTC+0800 check ops status done ops_status:mysql-niyrwp-upgrade-cmpv-rzd2l ns-lweur Upgrade mysql-niyrwp mysql Succeed 2/2 May 28,2025 13:09 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-upgrade-cmpv-rzd2l --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-upgrade-cmpv-rzd2l patched `kbcli cluster delete-ops --name mysql-niyrwp-upgrade-cmpv-rzd2l --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-upgrade-cmpv-rzd2l deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check readonly db_client batch data Success cluster update terminationPolicy WipeOut `kbcli cluster update mysql-niyrwp --termination-policy=WipeOut --namespace ns-lweur ` cluster.apps.kubeblocks.io/mysql-niyrwp updated check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql WipeOut Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-1 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-4-184.us-west-2.compute.internal/172.31.4.184 May 28,2025 12:22 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-13-207.us-west-2.compute.internal/172.31.13.207 May 28,2025 12:26 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kubectl get backupschedule -l app.kubernetes.io/instance=mysql-niyrwp ` `kubectl get backupschedule mysql-niyrwp-mysql-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched check backup status `kbcli cluster list-backups mysql-niyrwp --namespace ns-lweur ` No backups found in ns-lweur namespace. No backups found in ns-lweur namespace. backup_status:-- No backups found in ns-lweur namespace. backup_status:-- No backups found in ns-lweur namespace. backup_status:-- No backups found in ns-lweur namespace. backup_status:-- No backups found in ns-lweur namespace. backup_status:-- No backups found in ns-lweur namespace. backup_status:-- `kubectl get backupschedule -l app.kubernetes.io/instance=mysql-niyrwp ` `kubectl get backupschedule mysql-niyrwp-mysql-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched (no change) backup_status:mysql-niyrwp-xtrabackup-Running backup_status:mysql-niyrwp-xtrabackup-Running check backup status done backup_status:mysql-niyrwp-xtrabackup-20250528051200 ns-lweur mysql-niyrwp xtrabackup Completed 7781124 23s Delete May 28,2025 13:12 UTC+0800 May 28,2025 13:12 UTC+0800 Jun 05,2025 13:12 UTC+0800 `kubectl get backupschedule -l app.kubernetes.io/instance=mysql-niyrwp ` `kubectl get backupschedule mysql-niyrwp-mysql-backup-schedule -ojsonpath='***.spec.schedules[*].backupMethod***' ` backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched (no change) backupschedule.dataprotection.kubeblocks.io/mysql-niyrwp-mysql-backup-schedule patched (no change) cluster restore backup Error from server (NotFound): opsrequests.operations.kubeblocks.io "abk-niyrwp" not found `kbcli cluster describe-backup --names mysql-niyrwp-xtrabackup-20250528051200 --namespace ns-lweur ` Name: mysql-niyrwp-xtrabackup-20250528051200 Cluster: mysql-niyrwp Namespace: ns-lweur Spec: Method: xtrabackup Policy Name: mysql-niyrwp-mysql-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-mysql-niyrwp-xtrabackup-20250528051200-6d05d2d3 TargetPodName: mysql-niyrwp-mysql-1 Phase: Completed Start Time: May 28,2025 13:12 UTC+0800 Completion Time: May 28,2025 13:12 UTC+0800 Status: Phase: Completed Total Size: 7781124 ActionSet Name: mysql-xtrabackup-br Repository: backuprepo-kbcli-test Duration: 23s Expiration Time: Jun 05,2025 13:12 UTC+0800 Start Time: May 28,2025 13:12 UTC+0800 Completion Time: May 28,2025 13:12 UTC+0800 Path: /ns-lweur/mysql-niyrwp-6068df9f-1c16-4ac8-8adf-bd763066ac77/mysql/mysql-niyrwp-xtrabackup-20250528051200 Warning Events: `kbcli cluster restore abk-niyrwp --backup mysql-niyrwp-xtrabackup-20250528051200 --namespace ns-lweur ` Cluster abk-niyrwp created check cluster status `kbcli cluster list abk-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS abk-niyrwp ns-lweur mysql WipeOut Creating May 28,2025 13:12 UTC+0800 clusterdefinition.kubeblocks.io/name=mysql cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances abk-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME abk-niyrwp-mysql-0 ns-lweur abk-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-9-41.us-west-2.compute.internal/172.31.9.41 May 28,2025 13:13 UTC+0800 abk-niyrwp-mysql-1 ns-lweur abk-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 13:13 UTC+0800 abk-niyrwp-proxysql-0 ns-lweur abk-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-9-41.us-west-2.compute.internal/172.31.9.41 May 28,2025 13:13 UTC+0800 check pod status done check cluster role check cluster role done primary: abk-niyrwp-mysql-0;secondary: abk-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=abk-niyrwp` set secret: abk-niyrwp-mysql-account-root `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it abk-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kbcli cluster describe-backup --names mysql-niyrwp-xtrabackup-20250528051200 --namespace ns-lweur ` Name: mysql-niyrwp-xtrabackup-20250528051200 Cluster: mysql-niyrwp Namespace: ns-lweur Spec: Method: xtrabackup Policy Name: mysql-niyrwp-mysql-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-mysql-niyrwp-xtrabackup-20250528051200-6d05d2d3 TargetPodName: mysql-niyrwp-mysql-1 Phase: Completed Start Time: May 28,2025 13:12 UTC+0800 Completion Time: May 28,2025 13:12 UTC+0800 Status: Phase: Completed Total Size: 7781124 ActionSet Name: mysql-xtrabackup-br Repository: backuprepo-kbcli-test Duration: 23s Expiration Time: Jun 05,2025 13:12 UTC+0800 Start Time: May 28,2025 13:12 UTC+0800 Completion Time: May 28,2025 13:12 UTC+0800 Path: /ns-lweur/mysql-niyrwp-6068df9f-1c16-4ac8-8adf-bd763066ac77/mysql/mysql-niyrwp-xtrabackup-20250528051200 Warning Events: cluster connect `kubectl get secrets -l app.kubernetes.io/instance=abk-niyrwp` set secret: abk-niyrwp-mysql-account-root `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. *************************** 1. row *************************** Slave_IO_State: Waiting for source to send event Master_Host: abk-niyrwp-mysql-0.abk-niyrwp-mysql-headless Master_User: kbreplicator Master_Port: 3306 Connect_Retry: 60 Master_Log_File: abk-niyrwp-mysql-0-bin.000001 Read_Master_Log_Pos: 19405 Relay_Log_File: relay-bin.000002 Relay_Log_Pos: 19246 Relay_Master_Log_File: abk-niyrwp-mysql-0-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 19004 Relay_Log_Space: 19851 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: Yes Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: 7b5c4039-3b82-11f0-a349-9aceca6ab38a Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Replica has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: 7b5c4039-3b82-11f0-a349-9aceca6ab38a:1-48 Executed_Gtid_Set: 33dee2c2-3b75-11f0-b607-7ec94d11cac7:1-4068, 4e1a99ab-3b75-11f0-ad57-96b5a21536e9:1-1302, 7b5c4039-3b82-11f0-a349-9aceca6ab38a:1-48 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: Master_public_key_path: Get_master_public_key: 0 Network_Namespace: `echo " echo \"show variables like 'max_connections';\" | mysql -h127.0.0.1 -uroot -p94kc9Lx1R7 -P3306 " | kubectl exec -it abk-niyrwp-mysql-1 --namespace ns-lweur -- bash ` Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Variable_name Value max_connections 100 connect cluster Success set max_connections to 100 delete cluster abk-niyrwp `kbcli cluster delete abk-niyrwp --auto-approve --namespace ns-lweur ` Cluster abk-niyrwp deleted pod_info:abk-niyrwp-mysql-0 4/4 Running 0 85s abk-niyrwp-mysql-1 4/4 Running 0 62s abk-niyrwp-proxysql-0 1/1 Terminating 0 37s pod_info:abk-niyrwp-mysql-0 4/4 Running 0 106s abk-niyrwp-mysql-1 4/4 Running 0 83s abk-niyrwp-proxysql-0 1/1 Terminating 0 58s No resources found in ns-lweur namespace. delete cluster pod done No resources found in ns-lweur namespace. check cluster resource non-exist OK: pvc No resources found in ns-lweur namespace. delete cluster done No resources found in ns-lweur namespace. No resources found in ns-lweur namespace. No resources found in ns-lweur namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups mysql-niyrwp-xtrabackup-20250528051200 --namespace ns-lweur ` backup.dataprotection.kubeblocks.io/mysql-niyrwp-xtrabackup-20250528051200 patched `kbcli cluster delete-backup mysql-niyrwp --name mysql-niyrwp-xtrabackup-20250528051200 --force --auto-approve --namespace ns-lweur ` Backup mysql-niyrwp-xtrabackup-20250528051200 deleted cluster xtrabackup backup `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.name***"` `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.namespace***"` `kubectl get secrets kb-backuprepo-mb4vp -n kb-tgili -o jsonpath="***.data.accessKeyId***"` `kubectl get secrets kb-backuprepo-mb4vp -n kb-tgili -o jsonpath="***.data.secretAccessKey***"` KUBEBLOCKS NAMESPACE:kb-tgili get kubeblocks namespace done `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-tgili -o jsonpath="***.items[0].data.root-user***"` `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-tgili -o jsonpath="***.items[0].data.root-password***"` minio_user:kbclitest,minio_password:kbclitest,minio_endpoint:kbcli-test-minio.kb-tgili.svc.cluster.local:9000 list minio bucket kbcli-test `echo 'mc config host add minioserver http://kbcli-test-minio.kb-tgili.svc.cluster.local:9000 kbclitest kbclitest;mc ls minioserver' | kubectl exec -it kbcli-test-minio-5f4dfb568b-4g59b --namespace kb-tgili -- bash` Unable to use a TTY - input is not a terminal or the right kind of file list minio bucket done default backuprepo:backuprepo-kbcli-test exists `kbcli cluster backup mysql-niyrwp --method xtrabackup --namespace ns-lweur ` Backup backup-ns-lweur-mysql-niyrwp-20250528131538 created successfully, you can view the progress: kbcli cluster list-backups --name=backup-ns-lweur-mysql-niyrwp-20250528131538 -n ns-lweur check backup status `kbcli cluster list-backups mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION DELETION-POLICY CREATE-TIME COMPLETION-TIME EXPIRATION backup-ns-lweur-mysql-niyrwp-20250528131538 ns-lweur mysql-niyrwp xtrabackup Running Delete May 28,2025 13:15 UTC+0800 backup_status:mysql-niyrwp-xtrabackup-Running backup_status:mysql-niyrwp-xtrabackup-Running check backup status done backup_status:backup-ns-lweur-mysql-niyrwp-20250528131538 ns-lweur mysql-niyrwp xtrabackup Completed 7555695 11s Delete May 28,2025 13:15 UTC+0800 May 28,2025 13:15 UTC+0800 cluster restore backup Error from server (NotFound): opsrequests.operations.kubeblocks.io "abk-niyrwp" not found `kbcli cluster describe-backup --names backup-ns-lweur-mysql-niyrwp-20250528131538 --namespace ns-lweur ` Name: backup-ns-lweur-mysql-niyrwp-20250528131538 Cluster: mysql-niyrwp Namespace: ns-lweur Spec: Method: xtrabackup Policy Name: mysql-niyrwp-mysql-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-backup-ns-lweur-mysql-niyrwp-20250528131538-997c730 TargetPodName: mysql-niyrwp-mysql-1 Phase: Completed Start Time: May 28,2025 13:15 UTC+0800 Completion Time: May 28,2025 13:15 UTC+0800 Status: Phase: Completed Total Size: 7555695 ActionSet Name: mysql-xtrabackup-br Repository: backuprepo-kbcli-test Duration: 11s Start Time: May 28,2025 13:15 UTC+0800 Completion Time: May 28,2025 13:15 UTC+0800 Path: /ns-lweur/mysql-niyrwp-6068df9f-1c16-4ac8-8adf-bd763066ac77/mysql/backup-ns-lweur-mysql-niyrwp-20250528131538 Warning Events: `kbcli cluster restore abk-niyrwp --backup backup-ns-lweur-mysql-niyrwp-20250528131538 --namespace ns-lweur ` Cluster abk-niyrwp created check cluster status `kbcli cluster list abk-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS abk-niyrwp ns-lweur mysql WipeOut Creating May 28,2025 13:15 UTC+0800 clusterdefinition.kubeblocks.io/name=mysql cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances abk-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME abk-niyrwp-mysql-0 ns-lweur abk-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-9-41.us-west-2.compute.internal/172.31.9.41 May 28,2025 13:16 UTC+0800 abk-niyrwp-mysql-1 ns-lweur abk-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 13:16 UTC+0800 abk-niyrwp-proxysql-0 ns-lweur abk-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-5-153.us-west-2.compute.internal/172.31.5.153 May 28,2025 13:17 UTC+0800 check pod status done check cluster role check cluster role done primary: abk-niyrwp-mysql-0;secondary: abk-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=abk-niyrwp` set secret: abk-niyrwp-mysql-account-root `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it abk-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kbcli cluster describe-backup --names backup-ns-lweur-mysql-niyrwp-20250528131538 --namespace ns-lweur ` Name: backup-ns-lweur-mysql-niyrwp-20250528131538 Cluster: mysql-niyrwp Namespace: ns-lweur Spec: Method: xtrabackup Policy Name: mysql-niyrwp-mysql-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-backup-ns-lweur-mysql-niyrwp-20250528131538-997c730 TargetPodName: mysql-niyrwp-mysql-1 Phase: Completed Start Time: May 28,2025 13:15 UTC+0800 Completion Time: May 28,2025 13:15 UTC+0800 Status: Phase: Completed Total Size: 7555695 ActionSet Name: mysql-xtrabackup-br Repository: backuprepo-kbcli-test Duration: 11s Start Time: May 28,2025 13:15 UTC+0800 Completion Time: May 28,2025 13:15 UTC+0800 Path: /ns-lweur/mysql-niyrwp-6068df9f-1c16-4ac8-8adf-bd763066ac77/mysql/backup-ns-lweur-mysql-niyrwp-20250528131538 Warning Events: cluster connect `kubectl get secrets -l app.kubernetes.io/instance=abk-niyrwp` set secret: abk-niyrwp-mysql-account-root `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. *************************** 1. row *************************** Slave_IO_State: Waiting for source to send event Master_Host: abk-niyrwp-mysql-0.abk-niyrwp-mysql-headless Master_User: kbreplicator Master_Port: 3306 Connect_Retry: 60 Master_Log_File: abk-niyrwp-mysql-0-bin.000001 Read_Master_Log_Pos: 17400 Relay_Log_File: relay-bin.000002 Relay_Log_Pos: 17642 Relay_Master_Log_File: abk-niyrwp-mysql-0-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 17400 Relay_Log_Space: 17846 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: Yes Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: eefe6d54-3b82-11f0-b710-3272a9c7fa4f Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: waiting for handler commit Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: eefe6d54-3b82-11f0-b710-3272a9c7fa4f:1-43 Executed_Gtid_Set: 33dee2c2-3b75-11f0-b607-7ec94d11cac7:1-4275, 4e1a99ab-3b75-11f0-ad57-96b5a21536e9:1-1302, eefe6d54-3b82-11f0-b710-3272a9c7fa4f:1-43 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: Master_public_key_path: Get_master_public_key: 0 Network_Namespace: `echo " echo \"show variables like 'max_connections';\" | mysql -h127.0.0.1 -uroot -p94kc9Lx1R7 -P3306 " | kubectl exec -it abk-niyrwp-mysql-1 --namespace ns-lweur -- bash ` Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Variable_name Value max_connections 100 connect cluster Success set max_connections to 100 delete cluster abk-niyrwp `kbcli cluster delete abk-niyrwp --auto-approve --namespace ns-lweur ` Cluster abk-niyrwp deleted pod_info:abk-niyrwp-mysql-0 4/4 Running 0 80s abk-niyrwp-mysql-1 4/4 Running 0 58s abk-niyrwp-proxysql-0 1/1 Terminating 0 34s pod_info:abk-niyrwp-mysql-0 4/4 Running 0 101s abk-niyrwp-mysql-1 4/4 Running 0 79s abk-niyrwp-proxysql-0 1/1 Terminating 0 55s No resources found in ns-lweur namespace. delete cluster pod done No resources found in ns-lweur namespace. check cluster resource non-exist OK: pvc No resources found in ns-lweur namespace. delete cluster done No resources found in ns-lweur namespace. No resources found in ns-lweur namespace. No resources found in ns-lweur namespace. cluster rebulid instances apiVersion: operations.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: generateName: mysql-niyrwp-rebuildinstance- namespace: ns-lweur spec: type: RebuildInstance clusterName: mysql-niyrwp force: true rebuildFrom: - componentName: mysql instances: - name: mysql-niyrwp-mysql-1 backupName: backup-ns-lweur-mysql-niyrwp-20250528131538 inPlace: false restoreEnv: - name: BACKUP_FOR_STANDBY value: "true" check cluster status before ops check cluster status done cluster_status:Running `kubectl create -f test_ops_cluster_mysql-niyrwp.yaml` opsrequest.operations.kubeblocks.io/mysql-niyrwp-rebuildinstance-lbp7q created create test_ops_cluster_mysql-niyrwp.yaml Success `rm -rf test_ops_cluster_mysql-niyrwp.yaml` check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-rebuildinstance-lbp7q ns-lweur RebuildInstance mysql-niyrwp mysql Running 0/1 May 28,2025 13:18 UTC+0800 ops_status:mysql-niyrwp-rebuildinstance-lbp7q ns-lweur RebuildInstance mysql-niyrwp mysql Running 0/1 May 28,2025 13:18 UTC+0800 ops_status:mysql-niyrwp-rebuildinstance-lbp7q ns-lweur RebuildInstance mysql-niyrwp mysql Running 0/1 May 28,2025 13:18 UTC+0800 ops_status:mysql-niyrwp-rebuildinstance-lbp7q ns-lweur RebuildInstance mysql-niyrwp mysql Running 0/1 May 28,2025 13:18 UTC+0800 ops_status:mysql-niyrwp-rebuildinstance-lbp7q ns-lweur RebuildInstance mysql-niyrwp mysql Running 0/1 May 28,2025 13:18 UTC+0800 ops_status:mysql-niyrwp-rebuildinstance-lbp7q ns-lweur RebuildInstance mysql-niyrwp mysql Running 0/1 May 28,2025 13:18 UTC+0800 check ops status done ops_status:mysql-niyrwp-rebuildinstance-lbp7q ns-lweur RebuildInstance mysql-niyrwp mysql Succeed 1/1 May 28,2025 13:18 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-rebuildinstance-lbp7q --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-rebuildinstance-lbp7q patched `kbcli cluster delete-ops --name mysql-niyrwp-rebuildinstance-lbp7q --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-rebuildinstance-lbp7q deleted check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql WipeOut Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-2 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-9-41.us-west-2.compute.internal/172.31.9.41 May 28,2025 13:18 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 13:19 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-2 --namespace ns-lweur -- bash ` connect checking... check cluster connect done `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-2 --namespace ns-lweur -- bash ` check readonly db_client batch data Success cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups backup-ns-lweur-mysql-niyrwp-20250528131538 --namespace ns-lweur ` backup.dataprotection.kubeblocks.io/backup-ns-lweur-mysql-niyrwp-20250528131538 patched `kbcli cluster delete-backup mysql-niyrwp --name backup-ns-lweur-mysql-niyrwp-20250528131538 --force --auto-approve --namespace ns-lweur ` Backup backup-ns-lweur-mysql-niyrwp-20250528131538 deleted cluster xtrabackup backup `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.name***"` `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.namespace***"` `kubectl get secrets kb-backuprepo-mb4vp -n kb-tgili -o jsonpath="***.data.accessKeyId***"` `kubectl get secrets kb-backuprepo-mb4vp -n kb-tgili -o jsonpath="***.data.secretAccessKey***"` KUBEBLOCKS NAMESPACE:kb-tgili get kubeblocks namespace done `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-tgili -o jsonpath="***.items[0].data.root-user***"` `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-tgili -o jsonpath="***.items[0].data.root-password***"` minio_user:kbclitest,minio_password:kbclitest,minio_endpoint:kbcli-test-minio.kb-tgili.svc.cluster.local:9000 list minio bucket kbcli-test `echo 'mc config host add minioserver http://kbcli-test-minio.kb-tgili.svc.cluster.local:9000 kbclitest kbclitest;mc ls minioserver' | kubectl exec -it kbcli-test-minio-5f4dfb568b-4g59b --namespace kb-tgili -- bash` Unable to use a TTY - input is not a terminal or the right kind of file list minio bucket done default backuprepo:backuprepo-kbcli-test exists `kbcli cluster backup mysql-niyrwp --method xtrabackup --namespace ns-lweur ` Backup backup-ns-lweur-mysql-niyrwp-20250528132016 created successfully, you can view the progress: kbcli cluster list-backups --name=backup-ns-lweur-mysql-niyrwp-20250528132016 -n ns-lweur check backup status `kbcli cluster list-backups mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION DELETION-POLICY CREATE-TIME COMPLETION-TIME EXPIRATION backup-ns-lweur-mysql-niyrwp-20250528132016 ns-lweur mysql-niyrwp xtrabackup Running Delete May 28,2025 13:20 UTC+0800 backup_status:mysql-niyrwp-xtrabackup-Running backup_status:mysql-niyrwp-xtrabackup-Running check backup status done backup_status:backup-ns-lweur-mysql-niyrwp-20250528132016 ns-lweur mysql-niyrwp xtrabackup Completed 5912114 10s Delete May 28,2025 13:20 UTC+0800 May 28,2025 13:20 UTC+0800 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql `CREATE DATABASE IF NOT EXISTS kbclidb;CREATE TABLE IF NOT EXISTS kbclidb.msg (id INT UNSIGNED NOT NULL AUTO_INCREMENT,msg TEXT,time TIMESTAMP NULL,PRIMARY KEY (id)) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;insert into kbclidb.msg (msg, time) values ('kbcli-test-data-niyrwp0', CURRENT_TIMESTAMP);` Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql `insert into kbclidb.msg (msg, time) values ('kbcli-test-data-niyrwp1', CURRENT_TIMESTAMP);` Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql `insert into kbclidb.msg (msg, time) values ('kbcli-test-data-niyrwp2', CURRENT_TIMESTAMP);` Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql `insert into kbclidb.msg (msg, time) values ('kbcli-test-data-niyrwp3', CURRENT_TIMESTAMP);` Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. cluster xtrabackup-inc backup `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.name***"` `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.namespace***"` `kubectl get secrets kb-backuprepo-mb4vp -n kb-tgili -o jsonpath="***.data.accessKeyId***"` `kubectl get secrets kb-backuprepo-mb4vp -n kb-tgili -o jsonpath="***.data.secretAccessKey***"` KUBEBLOCKS NAMESPACE:kb-tgili get kubeblocks namespace done `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-tgili -o jsonpath="***.items[0].data.root-user***"` `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-tgili -o jsonpath="***.items[0].data.root-password***"` minio_user:kbclitest,minio_password:kbclitest,minio_endpoint:kbcli-test-minio.kb-tgili.svc.cluster.local:9000 list minio bucket kbcli-test `echo 'mc config host add minioserver http://kbcli-test-minio.kb-tgili.svc.cluster.local:9000 kbclitest kbclitest;mc ls minioserver' | kubectl exec -it kbcli-test-minio-5f4dfb568b-4g59b --namespace kb-tgili -- bash` Unable to use a TTY - input is not a terminal or the right kind of file list minio bucket done default backuprepo:backuprepo-kbcli-test exists `kbcli cluster backup mysql-niyrwp --method xtrabackup-inc --namespace ns-lweur ` Backup backup-ns-lweur-mysql-niyrwp-20250528132140 created successfully, you can view the progress: kbcli cluster list-backups --name=backup-ns-lweur-mysql-niyrwp-20250528132140 -n ns-lweur check backup status `kbcli cluster list-backups mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION DELETION-POLICY CREATE-TIME COMPLETION-TIME EXPIRATION backup-ns-lweur-mysql-niyrwp-20250528132016 ns-lweur mysql-niyrwp xtrabackup Completed 5912114 10s Delete May 28,2025 13:20 UTC+0800 May 28,2025 13:20 UTC+0800 backup-ns-lweur-mysql-niyrwp-20250528132140 ns-lweur mysql-niyrwp xtrabackup-inc Running Delete May 28,2025 13:21 UTC+0800 backup_status:mysql-niyrwp-xtrabackup-inc-Running backup_status:mysql-niyrwp-xtrabackup-inc-Running check backup status done backup_status:backup-ns-lweur-mysql-niyrwp-20250528132140 ns-lweur mysql-niyrwp xtrabackup-inc Completed 407995 10s Delete May 28,2025 13:21 UTC+0800 May 28,2025 13:21 UTC+0800 cluster restore backup Error from server (NotFound): opsrequests.operations.kubeblocks.io "abk-niyrwp" not found `kbcli cluster describe-backup --names backup-ns-lweur-mysql-niyrwp-20250528132140 --namespace ns-lweur ` Name: backup-ns-lweur-mysql-niyrwp-20250528132140 Cluster: mysql-niyrwp Namespace: ns-lweur Spec: Method: xtrabackup-inc Policy Name: mysql-niyrwp-mysql-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-backup-ns-lweur-mysql-niyrwp-20250528132140-efb4eb7 TargetPodName: mysql-niyrwp-mysql-2 Phase: Completed Start Time: May 28,2025 13:21 UTC+0800 Completion Time: May 28,2025 13:21 UTC+0800 Status: Phase: Completed Total Size: 407995 ActionSet Name: mysql-xtrabackup-inc-br Repository: backuprepo-kbcli-test Duration: 10s Start Time: May 28,2025 13:21 UTC+0800 Completion Time: May 28,2025 13:21 UTC+0800 Path: /ns-lweur/mysql-niyrwp-6068df9f-1c16-4ac8-8adf-bd763066ac77/mysql/backup-ns-lweur-mysql-niyrwp-20250528132140 Warning Events: `kbcli cluster restore abk-niyrwp --backup backup-ns-lweur-mysql-niyrwp-20250528132140 --namespace ns-lweur ` Cluster abk-niyrwp created check cluster status `kbcli cluster list abk-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS abk-niyrwp ns-lweur mysql WipeOut Creating May 28,2025 13:22 UTC+0800 clusterdefinition.kubeblocks.io/name=mysql cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances abk-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME abk-niyrwp-mysql-0 ns-lweur abk-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-8-216.us-west-2.compute.internal/172.31.8.216 May 28,2025 13:22 UTC+0800 abk-niyrwp-mysql-1 ns-lweur abk-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-5-43.us-west-2.compute.internal/172.31.5.43 May 28,2025 13:22 UTC+0800 abk-niyrwp-proxysql-0 ns-lweur abk-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 13:23 UTC+0800 check pod status done check cluster role check cluster role done primary: abk-niyrwp-mysql-0;secondary: abk-niyrwp-mysql-1 `kubectl get secrets -l app.kubernetes.io/instance=abk-niyrwp` set secret: abk-niyrwp-mysql-account-root `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it abk-niyrwp-mysql-1 --namespace ns-lweur -- bash ` check cluster connect done `kbcli cluster describe-backup --names backup-ns-lweur-mysql-niyrwp-20250528132140 --namespace ns-lweur ` Name: backup-ns-lweur-mysql-niyrwp-20250528132140 Cluster: mysql-niyrwp Namespace: ns-lweur Spec: Method: xtrabackup-inc Policy Name: mysql-niyrwp-mysql-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-backup-ns-lweur-mysql-niyrwp-20250528132140-efb4eb7 TargetPodName: mysql-niyrwp-mysql-2 Phase: Completed Start Time: May 28,2025 13:21 UTC+0800 Completion Time: May 28,2025 13:21 UTC+0800 Status: Phase: Completed Total Size: 407995 ActionSet Name: mysql-xtrabackup-inc-br Repository: backuprepo-kbcli-test Duration: 10s Start Time: May 28,2025 13:21 UTC+0800 Completion Time: May 28,2025 13:21 UTC+0800 Path: /ns-lweur/mysql-niyrwp-6068df9f-1c16-4ac8-8adf-bd763066ac77/mysql/backup-ns-lweur-mysql-niyrwp-20250528132140 Warning Events: `kubectl get secrets -l app.kubernetes.io/instance=abk-niyrwp` set secret: abk-niyrwp-mysql-account-root `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql `select * from kbclidb.msg;` Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. id msg time 1 kbcli-test-data-niyrwp0 2025-05-28 05:20:39 2 kbcli-test-data-niyrwp1 2025-05-28 05:20:47 3 kbcli-test-data-niyrwp2 2025-05-28 05:21:03 4 kbcli-test-data-niyrwp3 2025-05-28 05:21:18 Point-In-Time Recovery Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql cluster connect `kubectl get secrets -l app.kubernetes.io/instance=abk-niyrwp` set secret: abk-niyrwp-mysql-account-root `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets abk-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. *************************** 1. row *************************** Slave_IO_State: Waiting for source to send event Master_Host: abk-niyrwp-mysql-0.abk-niyrwp-mysql-headless Master_User: kbreplicator Master_Port: 3306 Connect_Retry: 60 Master_Log_File: abk-niyrwp-mysql-0-bin.000001 Read_Master_Log_Pos: 19806 Relay_Log_File: relay-bin.000002 Relay_Log_Pos: 20048 Relay_Master_Log_File: abk-niyrwp-mysql-0-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 19806 Relay_Log_Space: 20252 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: Yes Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: cc5ad592-3b83-11f0-949d-ded373b2c0c1 Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Replica has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: cc5ad592-3b83-11f0-949d-ded373b2c0c1:1-49 Executed_Gtid_Set: 33dee2c2-3b75-11f0-b607-7ec94d11cac7:1-4642, 4e1a99ab-3b75-11f0-ad57-96b5a21536e9:1-1296, cc5ad592-3b83-11f0-949d-ded373b2c0c1:1-49 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: Master_public_key_path: Get_master_public_key: 0 Network_Namespace: `echo " echo \"show variables like 'max_connections';\" | mysql -h127.0.0.1 -uroot -p94kc9Lx1R7 -P3306 " | kubectl exec -it abk-niyrwp-mysql-1 --namespace ns-lweur -- bash ` Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) Unable to use a TTY - input is not a terminal or the right kind of file Warning: arning] Using a password on the command line interface can be insecure. Variable_name Value max_connections 100 connect cluster Success set max_connections to 100 delete cluster abk-niyrwp `kbcli cluster delete abk-niyrwp --auto-approve --namespace ns-lweur ` Cluster abk-niyrwp deleted pod_info:abk-niyrwp-mysql-0 4/4 Running 0 94s abk-niyrwp-mysql-1 4/4 Running 0 70s abk-niyrwp-proxysql-0 1/1 Terminating 0 45s pod_info:abk-niyrwp-mysql-0 4/4 Running 0 115s abk-niyrwp-mysql-1 4/4 Running 0 91s abk-niyrwp-proxysql-0 1/1 Terminating 0 66s No resources found in ns-lweur namespace. delete cluster pod done No resources found in ns-lweur namespace. check cluster resource non-exist OK: pvc No resources found in ns-lweur namespace. delete cluster done No resources found in ns-lweur namespace. No resources found in ns-lweur namespace. No resources found in ns-lweur namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups backup-ns-lweur-mysql-niyrwp-20250528132016 --namespace ns-lweur ` backup.dataprotection.kubeblocks.io/backup-ns-lweur-mysql-niyrwp-20250528132016 patched `kbcli cluster delete-backup mysql-niyrwp --name backup-ns-lweur-mysql-niyrwp-20250528132016 --force --auto-approve --namespace ns-lweur ` Backup backup-ns-lweur-mysql-niyrwp-20250528132016 deleted `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups backup-ns-lweur-mysql-niyrwp-20250528132140 --namespace ns-lweur ` backup.dataprotection.kubeblocks.io/backup-ns-lweur-mysql-niyrwp-20250528132140 patched `kbcli cluster delete-backup mysql-niyrwp --name backup-ns-lweur-mysql-niyrwp-20250528132140 --force --auto-approve --namespace ns-lweur ` Backup backup-ns-lweur-mysql-niyrwp-20250528132140 deleted cluster list-logs `kbcli cluster list-logs mysql-niyrwp --namespace ns-lweur ` No log files found. Error from server (NotFound): pods "mysql-niyrwp-mysql-0" not found cluster logs `kbcli cluster logs mysql-niyrwp --tail 30 --namespace ns-lweur ` Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init) 2025-05-28T05:19:11.464801-00:00 0 [Note] [MY-011825] [Xtrabackup] Compressing and streaming 2025-05-28T05:19:11.464899-00:00 0 [Note] [MY-011825] [Xtrabackup] Done: Compressing and streaming file 2025-05-28T05:19:11.465007-00:00 0 [Note] [MY-011825] [Xtrabackup] Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS... 2025-05-28T05:19:11.473054-00:00 0 [Note] [MY-011825] [Xtrabackup] The latest check point (for incremental): '282795741' 2025-05-28T05:19:11.473103-00:00 0 [Note] [MY-011825] [Xtrabackup] Stopping log copying thread at LSN 284947112 2025-05-28T05:19:11.473526-00:00 1 [Note] [MY-011825] [Xtrabackup] >> log scanned up to (284951749) 2025-05-28T05:19:12.478215-00:00 0 [Note] [MY-011825] [Xtrabackup] Executing UNLOCK INSTANCE 2025-05-28T05:19:12.478861-00:00 0 [Note] [MY-011825] [Xtrabackup] All tables unlocked 2025-05-28T05:19:12.479138-00:00 0 [Note] [MY-011825] [Xtrabackup] Compressing and streaming ib_buffer_pool to 2025-05-28T05:19:12.480099-00:00 0 [Note] [MY-011825] [Xtrabackup] Done: Compressing and streaming ib_buffer_pool to 2025-05-28T05:19:12.480250-00:00 0 [Note] [MY-011825] [Xtrabackup] Backup created in directory '/xtrabackup_backupfiles/' 2025-05-28T05:19:12.480288-00:00 0 [Note] [MY-011825] [Xtrabackup] MySQL binlog position: filename 'mysql-niyrwp-mysql-0-bin.000030', position '237', GTID of the last change '33dee2c2-3b75-11f0-b607-7ec94d11cac7:1-4485,4e1a99ab-3b75-11f0-ad57-96b5a21536e9:1-1296' 2025-05-28T05:19:12.480388-00:00 0 [Note] [MY-011825] [Xtrabackup] Compressing and streaming 2025-05-28T05:19:12.480451-00:00 0 [Note] [MY-011825] [Xtrabackup] Done: Compressing and streaming file 2025-05-28T05:19:12.481073-00:00 0 [Note] [MY-011825] [Xtrabackup] Compressing and streaming 2025-05-28T05:19:12.481139-00:00 0 [Note] [MY-011825] [Xtrabackup] Done: Compressing and streaming file 2025-05-28T05:19:13.481470-00:00 0 [Note] [MY-011825] [Xtrabackup] Transaction log of lsn (282791057) to (284952488) was copied. 2025-05-28T05:19:13.644902-00:00 0 [Note] [MY-011825] [Xtrabackup] completed OK! 2025-05-28T05:19:27Z INFO HA This member is Cluster's leader 2025-05-28T05:19:27Z DEBUG HA Refresh leader ttl 2025-05-28T05:20:27Z INFO HA This member is Cluster's leader 2025-05-28T05:20:27Z DEBUG HA Refresh leader ttl 2025-05-28T05:21:27Z INFO HA This member is Cluster's leader 2025-05-28T05:21:27Z DEBUG HA Refresh leader ttl 2025-05-28T05:22:27Z INFO HA This member is Cluster's leader 2025-05-28T05:22:27Z DEBUG HA Refresh leader ttl 2025-05-28T05:23:27Z INFO HA This member is Cluster's leader 2025-05-28T05:23:27Z DEBUG HA Refresh leader ttl 2025-05-28T05:24:27Z INFO HA This member is Cluster's leader 2025-05-28T05:24:27Z DEBUG HA Refresh leader ttl cluster logs slow `kbcli cluster logs mysql-niyrwp --tail 30 --file-type=slow --namespace ns-lweur ` error: can't get file path pattern by type slow cluster logs error `kbcli cluster logs mysql-niyrwp --tail 30 --file-type=error --namespace ns-lweur ` error: can't get file path pattern by type error LB_TYPE is set to: internet cluster expose check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster expose mysql-niyrwp --auto-approve --force=true --type internet --enable false --components mysql --role-selector primary --namespace ns-lweur ` OpsRequest mysql-niyrwp-expose-6zg6t created successfully, you can view the progress: kbcli cluster describe-ops mysql-niyrwp-expose-6zg6t -n ns-lweur check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-expose-6zg6t ns-lweur Expose mysql-niyrwp mysql Running 0/1 May 28,2025 13:25 UTC+0800 check cluster status `kbcli cluster list mysql-niyrwp --show-labels --namespace ns-lweur ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mysql-niyrwp ns-lweur mysql WipeOut Running May 28,2025 11:36 UTC+0800 app.kubernetes.io/instance=mysql-niyrwp,clusterdefinition.kubeblocks.io/name=mysql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mysql-niyrwp --namespace ns-lweur ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mysql-niyrwp-mysql-0 ns-lweur mysql-niyrwp mysql Running primary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-2-51.us-west-2.compute.internal/172.31.2.51 May 28,2025 12:23 UTC+0800 mysql-niyrwp-mysql-2 ns-lweur mysql-niyrwp mysql Running secondary us-west-2a 200m / 200m 644245094400m / 644245094400m data:15Gi ip-172-31-9-41.us-west-2.compute.internal/172.31.9.41 May 28,2025 13:18 UTC+0800 mysql-niyrwp-proxysql-0 ns-lweur mysql-niyrwp proxysql Running us-west-2a 100m / 100m 512Mi / 512Mi ip-172-31-9-182.us-west-2.compute.internal/172.31.9.182 May 28,2025 13:19 UTC+0800 check pod status done check cluster role check cluster role done primary: mysql-niyrwp-mysql-0;secondary: mysql-niyrwp-mysql-2 `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check cluster connect `echo " echo \"SHOW SLAVE STATUS\G;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-2 --namespace ns-lweur -- bash ` check cluster connect done check ops status `kbcli cluster list-ops mysql-niyrwp --status all --namespace ns-lweur ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mysql-niyrwp-switchover-ndw4v ns-lweur Switchover mysql-niyrwp mysql Succeed 1/1 May 28,2025 11:54 UTC+0800 mysql-niyrwp-expose-6zg6t ns-lweur Expose mysql-niyrwp mysql Succeed 1/1 May 28,2025 13:25 UTC+0800 check ops status done ops_status:mysql-niyrwp-expose-6zg6t ns-lweur Expose mysql-niyrwp mysql Succeed 1/1 May 28,2025 13:25 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests.operations mysql-niyrwp-expose-6zg6t --namespace ns-lweur ` opsrequest.operations.kubeblocks.io/mysql-niyrwp-expose-6zg6t patched `kbcli cluster delete-ops --name mysql-niyrwp-expose-6zg6t --force --auto-approve --namespace ns-lweur ` OpsRequest mysql-niyrwp-expose-6zg6t deleted `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -hmysql-niyrwp-mysql.ns-lweur.svc.cluster.local -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-0 --namespace ns-lweur -- bash ` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mysql-niyrwp` set secret: mysql-niyrwp-mysql-account-root `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.username***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.password***"` `kubectl get secrets mysql-niyrwp-mysql-account-root -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:94kc9Lx1R7;DB_PORT:3306;DB_DATABASE:mysql check readonly db_client batch data count `echo "echo \"select count(*) from executions_loop.executions_loop_table;\" | mysql -P3306 -h127.0.0.1 -uroot -p'94kc9Lx1R7' " | kubectl exec -it mysql-niyrwp-mysql-2 --namespace ns-lweur -- bash ` check readonly db_client batch data Success delete cluster mysql-niyrwp `kbcli cluster delete mysql-niyrwp --auto-approve --namespace ns-lweur ` Cluster mysql-niyrwp deleted pod_info:mysql-niyrwp-mysql-0 4/4 Running 36 (15m ago) 62m mysql-niyrwp-mysql-2 4/4 Running 0 7m27s mysql-niyrwp-proxysql-0 1/1 Terminating 0 6m56s pod_info:mysql-niyrwp-mysql-0 4/4 Running 36 (16m ago) 63m mysql-niyrwp-mysql-2 4/4 Running 0 7m48s mysql-niyrwp-proxysql-0 1/1 Terminating 0 7m17s No resources found in ns-lweur namespace. delete cluster pod done No resources found in ns-lweur namespace. check cluster resource non-exist OK: pvc No resources found in ns-lweur namespace. delete cluster done No resources found in ns-lweur namespace. No resources found in ns-lweur namespace. No resources found in ns-lweur namespace. Mysql Test Suite All Done! --------------------------------------Mysql Proxysql (Topology = semisync-proxysql Replicas 2) Test Result-------------------------------------- [PASSED]|[Create]|[ComponentDefinition=mysql-8.0-1.0.0-alpha.0;ComponentVersion=mysql;ServiceVersion=8.0.30;]|[Description=Create a cluster with the specified component definition mysql-8.0-1.0.0-alpha.0 and component version mysql and service version 8.0.30] [PASSED]|[Connect]|[ComponentName=mysql]|[Description=Connect to the cluster] [PASSED]|[Expose]|[Enable=true;TYPE=internet;ComponentName=mysql]|[Description=Expose Enable the internet service with mysql component] [PASSED]|[Reconfiguring]|[ComponentName=mysql;binlog_expire_logs_seconds=691200]|[Description=Reconfiguring the cluster specify component mysql set binlog_expire_logs_seconds=691200] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [PASSED]|[Stop]|[-]|[Description=Stop the cluster] [PASSED]|[Start]|[-]|[Description=Start the cluster] [PASSED]|[Failover]|[HA=DNS Random;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience random IP addresses being returned by the DNS service for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to the DNS service returning random IP addresses.] [PASSED]|[SwitchOver]|[ComponentName=mysql]|[Description=SwitchOver the cluster specify component mysql] [PASSED]|[Bench]|[ComponentName=mysql]|[Description=Bench the cluster service with mysql component] [PASSED]|[Bench]|[HostType=LB;ComponentName=mysql]|[Description=Bench the cluster LB service with mysql component] [PASSED]|[Failover]|[HA=Network Corrupt Failover;Durations=2m;ComponentName=mysql]|[Description=] [PASSED]|[Failover]|[HA=Network Loss Failover;Durations=2m;ComponentName=mysql]|[Description=] [PASSED]|[Update]|[Monitor=true]|[Description=Update the cluster Monitor enable] [PASSED]|[Failover]|[HA=Pod Failure;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience failure for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to failure.] [PASSED]|[Failover]|[HA=Full CPU;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience CPU full either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high CPU load.] [PASSED]|[VolumeExpansion]|[ComponentName=mysql]|[Description=VolumeExpansion the cluster specify component mysql] [PASSED]|[Failover]|[HA=Delete Pod;ComponentName=mysql]|[Description=Simulates conditions where pods terminating forced/graceful thereby testing deployment sanity (replica availability & uninterrupted service) and recovery workflow of the application.] [PASSED]|[Failover]|[HA=Network Bandwidth Failover;Durations=2m;ComponentName=mysql]|[Description=] [PASSED]|[Failover]|[HA=Evicting Pod;ComponentName=mysql]|[Description=Simulates conditions where pods evicting either due to node drained thereby testing the application's resilience to unavailability of some replicas due to evicting.] [PASSED]|[Failover]|[HA=Kill 1;ComponentName=mysql]|[Description=Simulates conditions where process 1 killed either due to expected/undesired processes thereby testing the application's resilience to unavailability of some replicas due to abnormal termination signals.] [PASSED]|[Failover]|[HA=Network Delay;Durations=2m;ComponentName=mysql]|[Description=Simulates network delay fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to delay network.] [PASSED]|[VerticalScaling]|[ComponentName=mysql]|[Description=VerticalScaling the cluster specify component mysql] [PASSED]|[HorizontalScaling Out]|[ComponentName=mysql]|[Description=HorizontalScaling Out the cluster specify component mysql] [PASSED]|[HorizontalScaling In]|[ComponentName=mysql]|[Description=HorizontalScaling In the cluster specify component mysql] [PASSED]|[Failover]|[HA=Network Partition;Durations=2m;ComponentName=mysql]|[Description=Simulates network partition fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to partition network.] [PASSED]|[Failover]|[HA=DNS Error;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience DNS service errors for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to DNS service errors.] [PASSED]|[Failover]|[HA=Network Duplicate;Durations=2m;ComponentName=mysql]|[Description=Simulates network duplicate fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to duplicate network.] [PASSED]|[Failover]|[HA=Time Offset;Durations=2m;ComponentName=mysql]|[Description=Simulates a time offset scenario thereby testing the application's resilience to potential slowness/unavailability of some replicas due to time offset.] [PASSED]|[Failover]|[HA=OOM;Durations=2m;ComponentName=mysql]|[Description=Simulates conditions where pods experience OOM either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Memory load.] [PASSED]|[Failover]|[HA=Connection Stress;ComponentName=mysql]|[Description=Simulates conditions where pods experience connection stress either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Connection load.] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.30;ComponentVersionTo=8.0.31]|[Description=Upgrade the cluster specify component mysql service version from 8.0.30 to 8.0.31] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.31;ComponentVersionTo=8.0.32]|[Description=Upgrade the cluster specify component mysql service version from 8.0.31 to 8.0.32] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.32;ComponentVersionTo=8.0.33]|[Description=Upgrade the cluster specify component mysql service version from 8.0.32 to 8.0.33] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.33;ComponentVersionTo=8.0.34]|[Description=Upgrade the cluster specify component mysql service version from 8.0.33 to 8.0.34] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.34;ComponentVersionTo=8.0.35]|[Description=Upgrade the cluster specify component mysql service version from 8.0.34 to 8.0.35] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.35;ComponentVersionTo=8.0.39]|[Description=Upgrade the cluster specify component mysql service version from 8.0.35 to 8.0.39] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.39;ComponentVersionTo=8.0.35]|[Description=Upgrade the cluster specify component mysql service version from 8.0.39 to 8.0.35] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.35;ComponentVersionTo=8.0.36]|[Description=Upgrade the cluster specify component mysql service version from 8.0.35 to 8.0.36] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.36;ComponentVersionTo=8.0.39]|[Description=Upgrade the cluster specify component mysql service version from 8.0.36 to 8.0.39] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.39;ComponentVersionTo=8.0.36]|[Description=Upgrade the cluster specify component mysql service version from 8.0.39 to 8.0.36] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.36;ComponentVersionTo=8.0.37]|[Description=Upgrade the cluster specify component mysql service version from 8.0.36 to 8.0.37] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.37;ComponentVersionTo=8.0.39]|[Description=Upgrade the cluster specify component mysql service version from 8.0.37 to 8.0.39] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.39;ComponentVersionTo=8.0.37]|[Description=Upgrade the cluster specify component mysql service version from 8.0.39 to 8.0.37] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.37;ComponentVersionTo=8.0.38]|[Description=Upgrade the cluster specify component mysql service version from 8.0.37 to 8.0.38] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.38;ComponentVersionTo=8.0.39]|[Description=Upgrade the cluster specify component mysql service version from 8.0.38 to 8.0.39] [PASSED]|[Upgrade]|[ComponentName=mysql;ComponentVersionFrom=8.0.39;ComponentVersionTo=8.0.38]|[Description=Upgrade the cluster specify component mysql service version from 8.0.39 to 8.0.38] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Backup]|[Schedule=true;BackupMethod=xtrabackup]|[Description=The cluster Schedule xtrabackup Backup] [PASSED]|[Restore]|[Schedule=true;BackupMethod=xtrabackup]|[Description=The cluster Schedule xtrabackup Restore] [PASSED]|[Connect]|[ComponentName=mysql]|[Description=Connect to the cluster] [PASSED]|[Delete Restore Cluster]|[Schedule=true;BackupMethod=xtrabackup]|[Description=Delete the Schedule xtrabackup restore cluster] [PASSED]|[Backup]|[BackupMethod=xtrabackup]|[Description=The cluster xtrabackup Backup] [PASSED]|[Restore]|[BackupMethod=xtrabackup]|[Description=The cluster xtrabackup Restore] [PASSED]|[Connect]|[ComponentName=mysql]|[Description=Connect to the cluster] [PASSED]|[Delete Restore Cluster]|[BackupMethod=xtrabackup]|[Description=Delete the xtrabackup restore cluster] [PASSED]|[RebuildInstance]|[ComponentName=mysql]|[Description=Rebuild the cluster instance specify component mysql] [PASSED]|[Backup]|[BackupMethod=xtrabackup]|[Description=The cluster xtrabackup Backup] [PASSED]|[Backup]|[BackupMethod=xtrabackup-inc]|[Description=The cluster xtrabackup-inc Backup] [PASSED]|[Restore Increment]|[BackupMethod=xtrabackup]|[Description=The cluster xtrabackup Restore Increment] [PASSED]|[Connect]|[ComponentName=mysql]|[Description=Connect to the cluster] [PASSED]|[Delete Restore Cluster]|[BackupMethod=xtrabackup]|[Description=Delete the xtrabackup restore cluster] [PASSED]|[Expose]|[Disable=true;TYPE=internet;ComponentName=mysql]|[Description=Expose Disable the internet service with mysql component] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]