CURRENT_TEST_DIR:/Users/huangzhangshu/projects/apecloud/github/testinfra/test/kbcli source commons files source engines files source kubeblocks files source kubedb files CLUSTER_NAME: kbcli is detected: /usr/local/bin/kbcli Kubernetes Env: v1.30.4-vke.4 check snapshot controller check snapshot controller done vke kb-default-sc found vke default-vsc found POD_RESOURCES: No resources found found default storage class: ebs-ssd KubeBlocks version is:1.0.1 skip upgrade KubeBlocks current KubeBlocks version: 1.0.1 helm repo chaos-mesh https://charts.chaos-mesh.org exists chaos mesh already installed check component definition set component name:mssql set component version set component version:mssql set service versions:2022.19.0,2022.14.0 set service versions sorted:2022.14.0,2022.19.0 set mssql component definition set mssql component definition mssql-2022-1.0.1 REPORT_COUNT 0: 0 set replicas first:3,2022.14.0|3,2022.19.0 set replicas third:3,2022.14.0 set replicas fourth:3,2022.14.0 set minimum cmpv service version set minimum cmpv service version replicas:3,2022.14.0 REPORT_COUNT:1 CLUSTER_TOPOLOGY: set mssql component definition set mssql component definition mssql-2022-1.0.1 LIMIT_CPU:0.5 LIMIT_MEMORY:2 storage size: 20 CLUSTER_NAME:mssql-bxxmpv No resources found in default namespace. pod_info: termination_policy:DoNotTerminate create 3 replica DoNotTerminate mssql cluster check component definition set component definition by component version check cmpd by labels set component definition1: mssql-2022-1.0.1 by component version:mssql `kubectl apply -f test_create_mssql-bxxmpv.yaml` secret/mssql-bxxmpv-certificates created cluster.apps.kubeblocks.io/mssql-bxxmpv created apply test_create_mssql-bxxmpv.yaml Success `rm -rf test_create_mssql-bxxmpv.yaml` check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Creating Sep 12,2025 14:29 UTC+0800 clusterdefinition.kubeblocks.io/name=mssql cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 14:29 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 14:29 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.12/192.168.0.12 Sep 12,2025 14:29 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-0;secondary: mssql-bxxmpv-mssql-1 mssql-bxxmpv-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: check pod mssql-bxxmpv-mssql-0 container_name mssql exist password so6N5YwQk4 check pod mssql-bxxmpv-mssql-0 container_name exporter exist password so6N5YwQk4 check pod mssql-bxxmpv-mssql-0 container_name kbagent exist password so6N5YwQk4 check pod mssql-bxxmpv-mssql-0 container_name config-manager exist password so6N5YwQk4 No container logs contain secret password. describe cluster `kbcli cluster describe mssql-bxxmpv --namespace default ` Name: mssql-bxxmpv Created Time: Sep 12,2025 14:29 UTC+0800 NAMESPACE CLUSTER-DEFINITION TOPOLOGY STATUS TERMINATION-POLICY default mssql cluster Running DoNotTerminate Endpoints: COMPONENT INTERNAL EXTERNAL mssql mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local:1433 mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local:1433 Topology: COMPONENT SERVICE-VERSION INSTANCE ROLE STATUS AZ NODE CREATED-TIME mssql 2022.14.0 mssql-bxxmpv-mssql-0 primary Running cn-guangzhou-b 192.168.0.13/192.168.0.13 Sep 12,2025 14:29 UTC+0800 mssql 2022.14.0 mssql-bxxmpv-mssql-1 secondary Running cn-guangzhou-b 192.168.0.50/192.168.0.50 Sep 12,2025 14:29 UTC+0800 mssql 2022.14.0 mssql-bxxmpv-mssql-2 secondary Running cn-guangzhou-b 192.168.0.12/192.168.0.12 Sep 12,2025 14:29 UTC+0800 Resources Allocation: COMPONENT INSTANCE-TEMPLATE CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS mssql 500m / 500m 2Gi / 2Gi data:20Gi ebs-ssd Images: COMPONENT COMPONENT-DEFINITION IMAGE mssql mssql-2022-1.0.1 apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/mssql:2022-CU14 apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/prometheus-mssql-exporter:1.3.2 apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/mssql:2022-CU19 Data Protection: BACKUP-REPO AUTO-BACKUP BACKUP-SCHEDULE BACKUP-METHOD BACKUP-RETENTION RECOVERABLE-TIME Show cluster events: kbcli cluster list-events -n default mssql-bxxmpv `kbcli cluster label mssql-bxxmpv app.kubernetes.io/instance- --namespace default ` label "app.kubernetes.io/instance" not found. `kbcli cluster label mssql-bxxmpv app.kubernetes.io/instance=mssql-bxxmpv --namespace default ` `kbcli cluster label mssql-bxxmpv --list --namespace default ` NAME NAMESPACE LABELS mssql-bxxmpv default app.kubernetes.io/instance=mssql-bxxmpv clusterdefinition.kubeblocks.io/name=mssql label cluster app.kubernetes.io/instance=mssql-bxxmpv Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=mssql-bxxmpv --namespace default ` `kbcli cluster label mssql-bxxmpv --list --namespace default ` NAME NAMESPACE LABELS mssql-bxxmpv default app.kubernetes.io/instance=mssql-bxxmpv case.name=kbcli.test1 clusterdefinition.kubeblocks.io/name=mssql label cluster case.name=kbcli.test1 Success `kbcli cluster label mssql-bxxmpv case.name=kbcli.test2 --overwrite --namespace default ` `kbcli cluster label mssql-bxxmpv --list --namespace default ` NAME NAMESPACE LABELS mssql-bxxmpv default app.kubernetes.io/instance=mssql-bxxmpv case.name=kbcli.test2 clusterdefinition.kubeblocks.io/name=mssql label cluster case.name=kbcli.test2 Success `kbcli cluster label mssql-bxxmpv case.name- --namespace default ` `kbcli cluster label mssql-bxxmpv --list --namespace default ` NAME NAMESPACE LABELS mssql-bxxmpv default app.kubernetes.io/instance=mssql-bxxmpv clusterdefinition.kubeblocks.io/name=mssql delete cluster label case.name Success cluster connect `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"SELECT ag.name AS AvailabilityGroupName,ar.replica_server_name AS ReplicaName,ars.role_desc AS Role,ars.synchronization_health_desc AS SyncHealth FROM sys.availability_groups ag JOIN sys.availability_replicas ar ON ag.group_id = ar.group_id JOIN sys.dm_hadr_availability_replica_states ars ON ar.replica_id = ars.replica_id;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash ` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file AvailabilityGroupName ReplicaName Role SyncHealth -------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------ ------------------------------------------------------------ ag1 mssql-bxxmpv-mssql-0 PRIMARY HEALTHY ag1 mssql-bxxmpv-mssql-1 SECONDARY HEALTHY ag1 mssql-bxxmpv-mssql-2 SECONDARY HEALTHY (3 rows affected) connect cluster Success insert batch data by db client error: error from server (NotFound): pods "test-db-client-executionloop-mssql-bxxmpv" not found in namespace "default" `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge pods test-db-client-executionloop-mssql-bxxmpv --namespace default ` Error from server (NotFound): pods "test-db-client-executionloop-mssql-bxxmpv" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-executionloop-mssql-bxxmpv" not found `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. apiVersion: v1 kind: Pod metadata: name: test-db-client-executionloop-mssql-bxxmpv namespace: default spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/dbclient:test args: - "--host" - "mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local" - "--user" - "sa" - "--password" - "so6N5YwQk4" - "--port" - "1433" - "--dbtype" - "sqlserver" - "--test" - "executionloop" - "--duration" - "60" - "--interval" - "1" restartPolicy: Never `kubectl apply -f test-db-client-executionloop-mssql-bxxmpv.yaml` pod/test-db-client-executionloop-mssql-bxxmpv created apply test-db-client-executionloop-mssql-bxxmpv.yaml Success `rm -rf test-db-client-executionloop-mssql-bxxmpv.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 0/1 ContainerCreating 0 5s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 0/1 ContainerCreating 0 9s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 0/1 ContainerCreating 0 14s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 0/1 ContainerCreating 0 19s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 0/1 ContainerCreating 0 25s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 0/1 ContainerCreating 0 30s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 0/1 ContainerCreating 0 35s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 1/1 Running 0 40s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 1/1 Running 0 45s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 1/1 Running 0 51s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 1/1 Running 0 56s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 1/1 Running 0 61s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 1/1 Running 0 66s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 1/1 Running 0 71s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 1/1 Running 0 77s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 1/1 Running 0 82s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 1/1 Running 0 87s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 1/1 Running 0 92s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 1/1 Running 0 98s check pod test-db-client-executionloop-mssql-bxxmpv status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-bxxmpv 0/1 Completed 0 103s check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 14:29 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 14:29 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.12/192.168.0.12 Sep 12,2025 14:29 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-0;secondary: mssql-bxxmpv-mssql-1 mssql-bxxmpv-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED --host mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local --user sa --password so6N5YwQk4 --port 1433 --dbtype sqlserver --test executionloop --duration 60 --interval 1 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider] Execution loop start: create databases executions_loop IF NOT EXISTS (SELECT * FROM sys.databases WHERE name = 'executions_loop') CREATE DATABASE executions_loop drop table executions_loop_table IF OBJECT_ID('executions_loop..executions_loop_table', 'U') IS NOT NULL DROP TABLE executions_loop..executions_loop_table create table executions_loop_table Execution loop start: INSERT INTO executions_loop..executions_loop_table (value, tinyint_col, smallint_col, int_col, bigint_col, float_col, double_col, decimal_col, date_col, time_col, datetime_col, timestamp_col, year_col, char_col, text_col, blob_col, binary_col, varbinary_col, enum_col, set_col) VALUES ('executions_loop_test_1', 83, 3501, -30192046, 3375314064269479396, 0.5806735239094858, 0.44889375595443204, 56.38476627813939, '2025-09-12', '06:37:20', '2025-09-12 06:37:20.941', SYSUTCDATETIME(), '1985', 'hGGQDaOj3S', 'hDsxyHEsQIyWNQf277b51arr5KX1lA9obGv2V2UG1zuATFGAg5hc6hGVX4mIwsvylJZaFva15fi1Kzuo7OsEybsW9nrqUuj5J07yoYgsRO1xAFCYqZdGP7uLkKqZjcUuj2n8pabVn1doH5S12vwn0ZrLqKiFqCxPNkPq1jpgU4IV4DrUCPbrLfVwIHloes8tDpzr9IADIlJE9leBUrYUsTX6rRWXaosRo1V8byxIINKe3HzTRoTQ2BKGv7e6mgQ', 0x1e4cce00f8743b3f16b0, 0x196077d7f3198d01d871, 0x5a959a465d5ead5c0da492ed0924636cefa1b5a79ebcc0378166240578b0a71ebf067a65226642dfb6255ba706d395325bdf2f8a0ca605a25c583d88673370020e2f4c32dee073689dbcc1b74ca41ff52e1bd4093ee2a3db938c49dfc4b04b69b6f60885fcc2711df90525a0df1ba43711f3904553ab20452439ac07943650cc388a39dc6ef5dfac3353a177e42f63e7fef5ad532e7421ca85d7307b3ec38210aa484dab2ae246d56b7c37cd25e9df94c6faf077f6bc08391e9fbc4a0f6e774636022e2ca810596ff3820bd890938a128bc6a74f98f7a82ccccfc4feb81f9e319986adfe66d29f1bd203e592b87e501f4603d00d407cbfce60cc5343cbda3d, 'Option3', 'Value1') [ 1s ] executions total: 1 successful: 1 failed: 0 disconnect: 0 [ 2s ] executions total: 367 successful: 367 failed: 0 disconnect: 0 Failed to execute query: com.microsoft.sqlserver.jdbc.SQLServerException: Unable to access database 'executions_loop' because it lacks a quorum of nodes for high availability. Try the operation again later. [2025-09-12 06:37:22.022] Connection error occurred! [2025-09-12 06:37:22.022] Connection error occurred! [2025-09-12 06:37:23.043] Connection successfully recovered! The connection was restored in 1021 milliseconds. [ 3s ] executions total: 373 successful: 372 failed: 1 disconnect: 1 [ 4s ] executions total: 810 successful: 809 failed: 1 disconnect: 1 [ 5s ] executions total: 1242 successful: 1241 failed: 1 disconnect: 1 [ 6s ] executions total: 1684 successful: 1683 failed: 1 disconnect: 1 [ 7s ] executions total: 2135 successful: 2134 failed: 1 disconnect: 1 [ 8s ] executions total: 2612 successful: 2611 failed: 1 disconnect: 1 [ 9s ] executions total: 3090 successful: 3089 failed: 1 disconnect: 1 [ 10s ] executions total: 3504 successful: 3503 failed: 1 disconnect: 1 [ 11s ] executions total: 3948 successful: 3947 failed: 1 disconnect: 1 [ 12s ] executions total: 4423 successful: 4422 failed: 1 disconnect: 1 [ 13s ] executions total: 4855 successful: 4854 failed: 1 disconnect: 1 [ 14s ] executions total: 5319 successful: 5318 failed: 1 disconnect: 1 [ 15s ] executions total: 5790 successful: 5789 failed: 1 disconnect: 1 [ 16s ] executions total: 6249 successful: 6248 failed: 1 disconnect: 1 [ 17s ] executions total: 6720 successful: 6719 failed: 1 disconnect: 1 [ 18s ] executions total: 7197 successful: 7196 failed: 1 disconnect: 1 [ 19s ] executions total: 7683 successful: 7682 failed: 1 disconnect: 1 [ 20s ] executions total: 8158 successful: 8157 failed: 1 disconnect: 1 [ 21s ] executions total: 8631 successful: 8630 failed: 1 disconnect: 1 [ 22s ] executions total: 9099 successful: 9098 failed: 1 disconnect: 1 [ 23s ] executions total: 9574 successful: 9573 failed: 1 disconnect: 1 [ 24s ] executions total: 10079 successful: 10078 failed: 1 disconnect: 1 [ 25s ] executions total: 10568 successful: 10567 failed: 1 disconnect: 1 [ 26s ] executions total: 11059 successful: 11058 failed: 1 disconnect: 1 [ 27s ] executions total: 11558 successful: 11557 failed: 1 disconnect: 1 [ 28s ] executions total: 11996 successful: 11995 failed: 1 disconnect: 1 [ 29s ] executions total: 12477 successful: 12476 failed: 1 disconnect: 1 [ 30s ] executions total: 12971 successful: 12970 failed: 1 disconnect: 1 [ 31s ] executions total: 13451 successful: 13450 failed: 1 disconnect: 1 [ 32s ] executions total: 13922 successful: 13921 failed: 1 disconnect: 1 [ 33s ] executions total: 14399 successful: 14398 failed: 1 disconnect: 1 [ 34s ] executions total: 14866 successful: 14865 failed: 1 disconnect: 1 [ 35s ] executions total: 15358 successful: 15357 failed: 1 disconnect: 1 [ 36s ] executions total: 15833 successful: 15832 failed: 1 disconnect: 1 [ 37s ] executions total: 16315 successful: 16314 failed: 1 disconnect: 1 [ 38s ] executions total: 16778 successful: 16777 failed: 1 disconnect: 1 [ 39s ] executions total: 17265 successful: 17264 failed: 1 disconnect: 1 [ 40s ] executions total: 17727 successful: 17726 failed: 1 disconnect: 1 [ 41s ] executions total: 18129 successful: 18128 failed: 1 disconnect: 1 [ 42s ] executions total: 18586 successful: 18585 failed: 1 disconnect: 1 [ 43s ] executions total: 19040 successful: 19039 failed: 1 disconnect: 1 [ 44s ] executions total: 19505 successful: 19504 failed: 1 disconnect: 1 [ 45s ] executions total: 19989 successful: 19988 failed: 1 disconnect: 1 [ 46s ] executions total: 20480 successful: 20479 failed: 1 disconnect: 1 [ 47s ] executions total: 20906 successful: 20905 failed: 1 disconnect: 1 [ 48s ] executions total: 21314 successful: 21313 failed: 1 disconnect: 1 [ 49s ] executions total: 21719 successful: 21718 failed: 1 disconnect: 1 [ 50s ] executions total: 22164 successful: 22163 failed: 1 disconnect: 1 [ 51s ] executions total: 22634 successful: 22633 failed: 1 disconnect: 1 [ 52s ] executions total: 23103 successful: 23102 failed: 1 disconnect: 1 [ 53s ] executions total: 23571 successful: 23570 failed: 1 disconnect: 1 [ 54s ] executions total: 24055 successful: 24054 failed: 1 disconnect: 1 [ 55s ] executions total: 24537 successful: 24536 failed: 1 disconnect: 1 [ 56s ] executions total: 25023 successful: 25022 failed: 1 disconnect: 1 [ 57s ] executions total: 25524 successful: 25523 failed: 1 disconnect: 1 [ 60s ] executions total: 25571 successful: 25570 failed: 1 disconnect: 1 Test Result: Total Executions: 25571 Successful Executions: 25570 Failed Executions: 1 Disconnection Counts: 1 Connection Information: Database Type: sqlserver Host: mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local Port: 1433 Database: Table: User: sa Org: Access Mode: mysql Test Type: executionloop Query: Duration: 60 seconds Interval: 1 seconds DB_CLIENT_BATCH_DATA_COUNT: 25570 `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge pods test-db-client-executionloop-mssql-bxxmpv --namespace default ` pod/test-db-client-executionloop-mssql-bxxmpv patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-executionloop-mssql-bxxmpv" force deleted `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. set db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` set db_client batch data Success cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart mssql-bxxmpv --auto-approve --force=true --namespace default ` OpsRequest mssql-bxxmpv-restart-42s6v created successfully, you can view the progress: kbcli cluster describe-ops mssql-bxxmpv-restart-42s6v -n default check ops status `kbcli cluster list-ops mssql-bxxmpv --status all --namespace default ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-bxxmpv-restart-42s6v default Restart mssql-bxxmpv mssql Running 0/3 Sep 12,2025 14:38 UTC+0800 check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Updating Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating [Error] check cluster status timeout --------------------------------------get cluster mssql-bxxmpv yaml-------------------------------------- `kubectl get cluster mssql-bxxmpv -o yaml --namespace default ` apiVersion: apps.kubeblocks.io/v1 kind: Cluster metadata: annotations: kubeblocks.io/crd-api-version: apps.kubeblocks.io/v1 kubeblocks.io/extra-env: '{"MSSQL_PID":"Enterprise","MSSQL_COLLATION":"Chinese_PRC_CI_AS"}' kubeblocks.io/ops-request: '[{"name":"mssql-bxxmpv-restart-42s6v","type":"Restart"}]' kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"apps.kubeblocks.io/v1","kind":"Cluster","metadata":{"annotations":{"kubeblocks.io/extra-env":"{\"MSSQL_PID\":\"Enterprise\",\"MSSQL_COLLATION\":\"Chinese_PRC_CI_AS\"}"},"name":"mssql-bxxmpv","namespace":"default"},"spec":{"clusterDef":"mssql","componentSpecs":[{"env":[{"name":"MSSQL_PRIVATE_ENCRYPTION_PASSWORD","valueFrom":{"secretKeyRef":{"key":"private_key_password","name":"mssql-bxxmpv-certificates"}}},{"name":"HOST_NETWORK","value":"false"},{"name":"DEFAULT_DB_NAME","value":"db1"},{"name":"IS_REMOTE_STANDBY","value":"false"},{"name":"REMOTE_SESSION_TIMEOUT_SECONDS","value":"30"}],"name":"mssql","replicas":3,"resources":{"limits":{"cpu":"500m","memory":"2Gi"},"requests":{"cpu":"500m","memory":"2Gi"}},"serviceVersion":"2022.14.0","volumeClaimTemplates":[{"name":"data","spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"20Gi"}},"storageClassName":null}}],"volumes":[{"name":"certificates","secret":{"secretName":"mssql-bxxmpv-certificates"}}]}],"terminationPolicy":"DoNotTerminate","topology":null}} creationTimestamp: "2025-09-12T06:29:18Z" finalizers: - cluster.kubeblocks.io/finalizer generation: 3 labels: app.kubernetes.io/instance: mssql-bxxmpv clusterdefinition.kubeblocks.io/name: mssql name: mssql-bxxmpv namespace: default resourceVersion: "93528851" uid: 063f21b8-59ba-4474-aa9e-dcbe9dba91d5 spec: clusterDef: mssql componentSpecs: - annotations: kubeblocks.io/restart: "2025-09-12T06:38:29Z" componentDef: mssql-2022-1.0.1 env: - name: MSSQL_PRIVATE_ENCRYPTION_PASSWORD valueFrom: secretKeyRef: key: private_key_password name: mssql-bxxmpv-certificates - name: HOST_NETWORK value: "false" - name: DEFAULT_DB_NAME value: db1 - name: IS_REMOTE_STANDBY value: "false" - name: REMOTE_SESSION_TIMEOUT_SECONDS value: "30" name: mssql replicas: 3 resources: limits: cpu: 500m memory: 2Gi requests: cpu: 500m memory: 2Gi serviceVersion: 2022.14.0 volumeClaimTemplates: - name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi volumes: - name: certificates secret: secretName: mssql-bxxmpv-certificates terminationPolicy: DoNotTerminate topology: cluster status: components: mssql: observedGeneration: 3 phase: Updating upToDate: true conditions: - lastTransitionTime: "2025-09-12T06:29:18Z" message: 'The operator has started the provisioning of Cluster: mssql-bxxmpv' observedGeneration: 3 reason: PreCheckSucceed status: "True" type: ProvisioningStarted - lastTransitionTime: "2025-09-12T06:29:18Z" message: Successfully applied for resources observedGeneration: 3 reason: ApplyResourcesSucceed status: "True" type: ApplyResources - lastTransitionTime: "2025-09-12T06:36:04Z" message: cluster mssql-bxxmpv is ready reason: ClusterReady status: "True" type: Ready observedGeneration: 3 phase: Updating ------------------------------------------------------------------------------------------------------------------ --------------------------------------describe cluster mssql-bxxmpv-------------------------------------- `kubectl describe cluster mssql-bxxmpv --namespace default ` Name: mssql-bxxmpv Namespace: default Labels: app.kubernetes.io/instance=mssql-bxxmpv clusterdefinition.kubeblocks.io/name=mssql Annotations: kubeblocks.io/crd-api-version: apps.kubeblocks.io/v1 kubeblocks.io/extra-env: {"MSSQL_PID":"Enterprise","MSSQL_COLLATION":"Chinese_PRC_CI_AS"} kubeblocks.io/ops-request: [{"name":"mssql-bxxmpv-restart-42s6v","type":"Restart"}] API Version: apps.kubeblocks.io/v1 Kind: Cluster Metadata: Creation Timestamp: 2025-09-12T06:29:18Z Finalizers: cluster.kubeblocks.io/finalizer Generation: 3 Resource Version: 93528851 UID: 063f21b8-59ba-4474-aa9e-dcbe9dba91d5 Spec: Cluster Def: mssql Component Specs: Annotations: kubeblocks.io/restart: 2025-09-12T06:38:29Z Component Def: mssql-2022-1.0.1 Env: Name: MSSQL_PRIVATE_ENCRYPTION_PASSWORD Value From: Secret Key Ref: Key: private_key_password Name: mssql-bxxmpv-certificates Name: HOST_NETWORK Value: false Name: DEFAULT_DB_NAME Value: db1 Name: IS_REMOTE_STANDBY Value: false Name: REMOTE_SESSION_TIMEOUT_SECONDS Value: 30 Name: mssql Replicas: 3 Resources: Limits: Cpu: 500m Memory: 2Gi Requests: Cpu: 500m Memory: 2Gi Service Version: 2022.14.0 Volume Claim Templates: Name: data Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 20Gi Volumes: Name: certificates Secret: Secret Name: mssql-bxxmpv-certificates Termination Policy: DoNotTerminate Topology: cluster Status: Components: Mssql: Observed Generation: 3 Phase: Updating Up To Date: true Conditions: Last Transition Time: 2025-09-12T06:29:18Z Message: The operator has started the provisioning of Cluster: mssql-bxxmpv Observed Generation: 3 Reason: PreCheckSucceed Status: True Type: ProvisioningStarted Last Transition Time: 2025-09-12T06:29:18Z Message: Successfully applied for resources Observed Generation: 3 Reason: ApplyResourcesSucceed Status: True Type: ApplyResources Last Transition Time: 2025-09-12T06:36:04Z Message: cluster mssql-bxxmpv is ready Reason: ClusterReady Status: True Type: Ready Observed Generation: 3 Phase: Updating Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ClusterComponentPhaseTransition 16m cluster-controller cluster component mssql is Creating Normal ClusterReady 9m54s cluster-controller cluster mssql-bxxmpv is ready Normal Running 9m54s cluster-controller Cluster: mssql-bxxmpv is ready, current phase is Running Normal PreCheckSucceed 7m29s (x3 over 16m) cluster-controller The operator has started the provisioning of Cluster: mssql-bxxmpv Normal ApplyResourcesSucceed 7m29s (x3 over 16m) cluster-controller Successfully applied for resources Normal ClusterComponentPhaseTransition 7m29s (x11 over 9m54s) cluster-controller cluster component mssql is Running Normal ClusterComponentPhaseTransition 7m29s (x4 over 7m29s) cluster-controller cluster component mssql is Updating ------------------------------------------------------------------------------------------------------------------ check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 14:29 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 14:45 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 14:38 UTC+0800 check pod status done check cluster status again cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status again done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED check ops status `kbcli cluster list-ops mssql-bxxmpv --status all --namespace default ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-bxxmpv-restart-42s6v default Restart mssql-bxxmpv mssql Succeed 3/3 Sep 12,2025 14:38 UTC+0800 check ops status done ops_status:mssql-bxxmpv-restart-42s6v default Restart mssql-bxxmpv mssql Succeed 3/3 Sep 12,2025 14:38 UTC+0800 `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge opsrequests.operations mssql-bxxmpv-restart-42s6v --namespace default ` opsrequest.operations.kubeblocks.io/mssql-bxxmpv-restart-42s6v patched `kbcli cluster delete-ops --name mssql-bxxmpv-restart-42s6v --force --auto-approve --namespace default ` OpsRequest mssql-bxxmpv-restart-42s6v deleted `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data Success test failover dnsrandom check cluster status before cluster-failover-dnsrandom check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge DNSChaos test-chaos-mesh-dnsrandom-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mssql-bxxmpv" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnsrandom-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-2 mode: all action: random duration: 2m `kubectl apply -f test-chaos-mesh-dnsrandom-mssql-bxxmpv.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnsrandom-mssql-bxxmpv created apply test-chaos-mesh-dnsrandom-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-dnsrandom-mssql-bxxmpv.yaml` dnsrandom chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 14:46 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 14:45 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 14:38 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge DNSChaos test-chaos-mesh-dnsrandom-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mssql-bxxmpv" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-2 failover dnsrandom Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data retry times: 1 check readonly db_client batch data Success test failover dnserror check cluster status before cluster-failover-dnserror check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge DNSChaos test-chaos-mesh-dnserror-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mssql-bxxmpv" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnserror-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-2 mode: all action: error duration: 2m `kubectl apply -f test-chaos-mesh-dnserror-mssql-bxxmpv.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnserror-mssql-bxxmpv created apply test-chaos-mesh-dnserror-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-dnserror-mssql-bxxmpv.yaml` dnserror chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 14:46 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 14:45 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 14:38 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge DNSChaos test-chaos-mesh-dnserror-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mssql-bxxmpv" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-2 failover dnserror Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data retry times: 1 check readonly db_client batch data Success test failover networkbandwidthover check cluster status before cluster-failover-networkbandwidthover check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mssql-bxxmpv" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkbandwidthover-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-2 action: bandwidth mode: all bandwidth: rate: '1bps' limit: 20971520 buffer: 10000 duration: 2m `kubectl apply -f test-chaos-mesh-networkbandwidthover-mssql-bxxmpv.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkbandwidthover-mssql-bxxmpv created apply test-chaos-mesh-networkbandwidthover-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-networkbandwidthover-mssql-bxxmpv.yaml` networkbandwidthover chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 14:46 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 14:45 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 14:38 UTC+0800 check pod status done check cluster role primary: mssql-bxxmpv-mssql-1 mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 check cluster role done primary: mssql-bxxmpv-mssql-1;secondary: mssql-bxxmpv-mssql-0 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Sqlcmd: Error: Microsoft ODBC Driver 18 for SQL Server : Login timeout expired. Sqlcmd: Error: Microsoft ODBC Driver 18 for SQL Server : TCP Provider: Error code 0x102. Sqlcmd: Error: Microsoft ODBC Driver 18 for SQL Server : A network-related or instance-specific error has occurred while establishing a connection to mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online.. command terminated with exit code 1 check mssql synchronization state... Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mssql-bxxmpv" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-1 failover networkbandwidthover Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data Success test failover networklossover check cluster status before cluster-failover-networklossover cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge NetworkChaos test-chaos-mesh-networklossover-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mssql-bxxmpv" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-1 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-mssql-bxxmpv.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-mssql-bxxmpv created apply test-chaos-mesh-networklossover-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-networklossover-mssql-bxxmpv.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 14:46 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 14:45 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 14:38 UTC+0800 check pod status done check cluster role primary: mssql-bxxmpv-mssql-1 mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 primary: mssql-bxxmpv-mssql-1 mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 primary: mssql-bxxmpv-mssql-1 mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 primary: mssql-bxxmpv-mssql-1 mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 primary: mssql-bxxmpv-mssql-1 mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 check cluster role done primary: mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file SYNCHRONIZING check mssql synchronization state... SYNCHRONIZING SYNCHRONIZED Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge NetworkChaos test-chaos-mesh-networklossover-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mssql-bxxmpv" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-2 failover networklossover Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data Success test failover podfailure check cluster status before cluster-failover-podfailure cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge PodChaos test-chaos-mesh-podfailure-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mssql-bxxmpv" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podfailure-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-2 mode: all action: pod-failure duration: 2m `kubectl apply -f test-chaos-mesh-podfailure-mssql-bxxmpv.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podfailure-mssql-bxxmpv created apply test-chaos-mesh-podfailure-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-podfailure-mssql-bxxmpv.yaml` podfailure chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Failed Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql cluster_status:Failed cluster_status:Failed cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 14:46 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 14:45 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 500m / 500m 2Gi / 2Gi data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 14:38 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-1;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge PodChaos test-chaos-mesh-podfailure-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mssql-bxxmpv" force deleted Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-1 failover podfailure Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data Success check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale mssql-bxxmpv --auto-approve --force=true --components mssql --cpu 600m --memory 2.1Gi --namespace default ` OpsRequest mssql-bxxmpv-verticalscaling-cl2rb created successfully, you can view the progress: kbcli cluster describe-ops mssql-bxxmpv-verticalscaling-cl2rb -n default check ops status `kbcli cluster list-ops mssql-bxxmpv --status all --namespace default ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-bxxmpv-verticalscaling-cl2rb default VerticalScaling mssql-bxxmpv mssql Running 0/3 Sep 12,2025 15:03 UTC+0800 check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Updating Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 15:03 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.12/192.168.0.12 Sep 12,2025 15:04 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:03 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-0;secondary: mssql-bxxmpv-mssql-1 mssql-bxxmpv-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED check ops status `kbcli cluster list-ops mssql-bxxmpv --status all --namespace default ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-bxxmpv-verticalscaling-cl2rb default VerticalScaling mssql-bxxmpv mssql Succeed 3/3 Sep 12,2025 15:03 UTC+0800 check ops status done ops_status:mssql-bxxmpv-verticalscaling-cl2rb default VerticalScaling mssql-bxxmpv mssql Succeed 3/3 Sep 12,2025 15:03 UTC+0800 `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge opsrequests.operations mssql-bxxmpv-verticalscaling-cl2rb --namespace default ` opsrequest.operations.kubeblocks.io/mssql-bxxmpv-verticalscaling-cl2rb patched `kbcli cluster delete-ops --name mssql-bxxmpv-verticalscaling-cl2rb --force --auto-approve --namespace default ` OpsRequest mssql-bxxmpv-verticalscaling-cl2rb deleted `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` check readonly db_client batch data Success test failover networkdelay check cluster status before cluster-failover-networkdelay check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge NetworkChaos test-chaos-mesh-networkdelay-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mssql-bxxmpv" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkdelay-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-0 mode: all action: delay delay: latency: 2000ms correlation: '100' jitter: 0ms direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkdelay-mssql-bxxmpv.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkdelay-mssql-bxxmpv created apply test-chaos-mesh-networkdelay-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-networkdelay-mssql-bxxmpv.yaml` networkdelay chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 15:03 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.12/192.168.0.12 Sep 12,2025 15:04 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:03 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-0;secondary: mssql-bxxmpv-mssql-1 mssql-bxxmpv-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge NetworkChaos test-chaos-mesh-networkdelay-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mssql-bxxmpv" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-0 failover networkdelay Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` check readonly db_client batch data Success test switchover cluster promote check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster promote mssql-bxxmpv --auto-approve --force=true --instance mssql-bxxmpv-mssql-0 --candidate mssql-bxxmpv-mssql-1 --namespace default ` OpsRequest mssql-bxxmpv-switchover-wlj9b created successfully, you can view the progress: kbcli cluster describe-ops mssql-bxxmpv-switchover-wlj9b -n default check ops status `kbcli cluster list-ops mssql-bxxmpv --status all --namespace default ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-bxxmpv-switchover-wlj9b default Switchover mssql-bxxmpv mssql-bxxmpv-mssql Running 0/1 Sep 12,2025 15:09 UTC+0800 check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 15:03 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.12/192.168.0.12 Sep 12,2025 15:04 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:03 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-1;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED check ops status `kbcli cluster list-ops mssql-bxxmpv --status all --namespace default ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-bxxmpv-switchover-wlj9b default Switchover mssql-bxxmpv mssql-bxxmpv-mssql Succeed 1/1 Sep 12,2025 15:09 UTC+0800 check ops status done ops_status:mssql-bxxmpv-switchover-wlj9b default Switchover mssql-bxxmpv mssql-bxxmpv-mssql Succeed 1/1 Sep 12,2025 15:09 UTC+0800 `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge opsrequests.operations mssql-bxxmpv-switchover-wlj9b --namespace default ` opsrequest.operations.kubeblocks.io/mssql-bxxmpv-switchover-wlj9b patched `kbcli cluster delete-ops --name mssql-bxxmpv-switchover-wlj9b --force --auto-approve --namespace default ` OpsRequest mssql-bxxmpv-switchover-wlj9b deleted `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data Success switchover pod:mssql-bxxmpv-mssql-1 switchover success test failover podkill check cluster status before cluster-failover-podkill check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge PodChaos test-chaos-mesh-podkill-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podkill-mssql-bxxmpv" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podkill-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podkill-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-1 mode: all action: pod-kill `kubectl apply -f test-chaos-mesh-podkill-mssql-bxxmpv.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podkill-mssql-bxxmpv created apply test-chaos-mesh-podkill-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-podkill-mssql-bxxmpv.yaml` check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Updating Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 15:03 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.12/192.168.0.12 Sep 12,2025 15:10 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:03 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-0;secondary: mssql-bxxmpv-mssql-1 mssql-bxxmpv-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge PodChaos test-chaos-mesh-podkill-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podkill-mssql-bxxmpv" force deleted Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podkill-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-0 failover podkill Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` check readonly db_client batch data Success test failover networkbandwidthover check cluster status before cluster-failover-networkbandwidthover check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mssql-bxxmpv" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkbandwidthover-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-0 action: bandwidth mode: all bandwidth: rate: '1bps' limit: 20971520 buffer: 10000 duration: 2m `kubectl apply -f test-chaos-mesh-networkbandwidthover-mssql-bxxmpv.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkbandwidthover-mssql-bxxmpv created apply test-chaos-mesh-networkbandwidthover-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-networkbandwidthover-mssql-bxxmpv.yaml` networkbandwidthover chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 15:03 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.12/192.168.0.12 Sep 12,2025 15:10 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:03 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-1;secondary: mssql-bxxmpv-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mssql-bxxmpv" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-1 failover networkbandwidthover Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` check readonly db_client batch data Success cluster does not need to check monitor currently check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Failed Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 15:03 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.12/192.168.0.12 Sep 12,2025 15:10 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:03 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-1;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED test failover networkcorruptover check cluster status before cluster-failover-networkcorruptover check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mssql-bxxmpv" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorruptover-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-1 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorruptover-mssql-bxxmpv.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-mssql-bxxmpv created apply test-chaos-mesh-networkcorruptover-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-networkcorruptover-mssql-bxxmpv.yaml` networkcorruptover chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.13/192.168.0.13 Sep 12,2025 15:03 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.12/192.168.0.12 Sep 12,2025 15:10 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:03 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-1;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mssql-bxxmpv" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-0 failover networkcorruptover Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` check readonly db_client batch data Success cluster stop check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster stop mssql-bxxmpv --auto-approve --force=true --namespace default ` OpsRequest mssql-bxxmpv-stop-thrh8 created successfully, you can view the progress: kbcli cluster describe-ops mssql-bxxmpv-stop-thrh8 -n default check ops status `kbcli cluster list-ops mssql-bxxmpv --status all --namespace default ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-bxxmpv-stop-thrh8 default Stop mssql-bxxmpv mssql Running 0/3 Sep 12,2025 15:18 UTC+0800 check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Stopping Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql cluster_status:Stopping cluster_status:Stopping check cluster status done cluster_status:Stopped check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME check pod status done check ops status `kbcli cluster list-ops mssql-bxxmpv --status all --namespace default ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-bxxmpv-stop-thrh8 default Stop mssql-bxxmpv mssql Succeed 3/3 Sep 12,2025 15:18 UTC+0800 check ops status done ops_status:mssql-bxxmpv-stop-thrh8 default Stop mssql-bxxmpv mssql Succeed 3/3 Sep 12,2025 15:18 UTC+0800 `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge opsrequests.operations mssql-bxxmpv-stop-thrh8 --namespace default ` opsrequest.operations.kubeblocks.io/mssql-bxxmpv-stop-thrh8 patched `kbcli cluster delete-ops --name mssql-bxxmpv-stop-thrh8 --force --auto-approve --namespace default ` OpsRequest mssql-bxxmpv-stop-thrh8 deleted cluster start check cluster status before ops check cluster status done cluster_status:Stopped `kbcli cluster start mssql-bxxmpv --force=true --namespace default ` OpsRequest mssql-bxxmpv-start-mlp2t created successfully, you can view the progress: kbcli cluster describe-ops mssql-bxxmpv-start-mlp2t -n default check ops status `kbcli cluster list-ops mssql-bxxmpv --status all --namespace default ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-bxxmpv-start-mlp2t default Start mssql-bxxmpv mssql Running 0/3 Sep 12,2025 15:18 UTC+0800 check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Updating Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED check ops status `kbcli cluster list-ops mssql-bxxmpv --status all --namespace default ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-bxxmpv-start-mlp2t default Start mssql-bxxmpv mssql Succeed 3/3 Sep 12,2025 15:18 UTC+0800 check ops status done ops_status:mssql-bxxmpv-start-mlp2t default Start mssql-bxxmpv mssql Succeed 3/3 Sep 12,2025 15:18 UTC+0800 `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge opsrequests.operations mssql-bxxmpv-start-mlp2t --namespace default ` opsrequest.operations.kubeblocks.io/mssql-bxxmpv-start-mlp2t patched `kbcli cluster delete-ops --name mssql-bxxmpv-start-mlp2t --force --auto-approve --namespace default ` OpsRequest mssql-bxxmpv-start-mlp2t deleted `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data retry times: 1 check readonly db_client batch data Success test failover timeoffset check cluster status before cluster-failover-timeoffset check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge TimeChaos test-chaos-mesh-timeoffset-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mssql-bxxmpv" not found Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: TimeChaos metadata: name: test-chaos-mesh-timeoffset-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-2 mode: all timeOffset: '-10m' clockIds: - CLOCK_REALTIME duration: 2m `kubectl apply -f test-chaos-mesh-timeoffset-mssql-bxxmpv.yaml` timechaos.chaos-mesh.org/test-chaos-mesh-timeoffset-mssql-bxxmpv created apply test-chaos-mesh-timeoffset-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-timeoffset-mssql-bxxmpv.yaml` timeoffset chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Failed Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql cluster_status:Failed check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge TimeChaos test-chaos-mesh-timeoffset-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mssql-bxxmpv" force deleted Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-2 failover timeoffset Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data Success test failover fullcpu check cluster status before cluster-failover-fullcpu check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge StressChaos test-chaos-mesh-fullcpu-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mssql-bxxmpv" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-fullcpu-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-2 mode: all stressors: cpu: workers: 100 load: 100 duration: 2m `kubectl apply -f test-chaos-mesh-fullcpu-mssql-bxxmpv.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-fullcpu-mssql-bxxmpv created apply test-chaos-mesh-fullcpu-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-fullcpu-mssql-bxxmpv.yaml` fullcpu chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge StressChaos test-chaos-mesh-fullcpu-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mssql-bxxmpv" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-2 failover fullcpu Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data Success test failover networkduplicate check cluster status before cluster-failover-networkduplicate check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mssql-bxxmpv" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkduplicate-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-2 mode: all action: duplicate duplicate: duplicate: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkduplicate-mssql-bxxmpv.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkduplicate-mssql-bxxmpv created apply test-chaos-mesh-networkduplicate-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-networkduplicate-mssql-bxxmpv.yaml` networkduplicate chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mssql-bxxmpv" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-2 failover networkduplicate Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data Success test failover oom check cluster status before cluster-failover-oom check cluster status done cluster_status:Running `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge StressChaos test-chaos-mesh-oom-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-bxxmpv" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-2 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-mssql-bxxmpv.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mssql-bxxmpv created apply test-chaos-mesh-oom-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-oom-mssql-bxxmpv.yaml` check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge StressChaos test-chaos-mesh-oom-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-bxxmpv" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-bxxmpv" not found check failover pod name failover pod name:mssql-bxxmpv-mssql-2 checking failover... `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge StressChaos test-chaos-mesh-oom-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-bxxmpv" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-bxxmpv" not found `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge StressChaos test-chaos-mesh-oom-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-bxxmpv" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-bxxmpv" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-mssql-bxxmpv namespace: default spec: selector: namespaces: - default labelSelectors: apps.kubeblocks.io/pod-name: mssql-bxxmpv-mssql-2 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-mssql-bxxmpv.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mssql-bxxmpv created apply test-chaos-mesh-oom-mssql-bxxmpv.yaml Success `rm -rf test-chaos-mesh-oom-mssql-bxxmpv.yaml` check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Updating Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:20Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-1;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge StressChaos test-chaos-mesh-oom-mssql-bxxmpv --namespace default ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-bxxmpv" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-bxxmpv" not found check failover pod name:mssql-bxxmpv-mssql-1 failover oom Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data Success `kubectl get pvc -l app.kubernetes.io/instance=mssql-bxxmpv,apps.kubeblocks.io/component-name=mssql,apps.kubeblocks.io/vct-name=data --namespace default ` cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in mssql-bxxmpv namespace. `kbcli cluster volume-expand mssql-bxxmpv --auto-approve --force=true --components mssql --volume-claim-templates data --storage 22Gi --namespace default ` OpsRequest mssql-bxxmpv-volumeexpansion-tb7k4 created successfully, you can view the progress: kbcli cluster describe-ops mssql-bxxmpv-volumeexpansion-tb7k4 -n default check ops status `kbcli cluster list-ops mssql-bxxmpv --status all --namespace default ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-bxxmpv-volumeexpansion-tb7k4 default VolumeExpansion mssql-bxxmpv mssql Running 0/3 Sep 12,2025 15:27 UTC+0800 check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Updating Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:22Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:22Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:22Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-1;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED No resources found in mssql-bxxmpv namespace. check ops status `kbcli cluster list-ops mssql-bxxmpv --status all --namespace default ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-bxxmpv-volumeexpansion-tb7k4 default VolumeExpansion mssql-bxxmpv mssql Succeed 3/3 Sep 12,2025 15:27 UTC+0800 check ops status done ops_status:mssql-bxxmpv-volumeexpansion-tb7k4 default VolumeExpansion mssql-bxxmpv mssql Succeed 3/3 Sep 12,2025 15:27 UTC+0800 `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge opsrequests.operations mssql-bxxmpv-volumeexpansion-tb7k4 --namespace default ` opsrequest.operations.kubeblocks.io/mssql-bxxmpv-volumeexpansion-tb7k4 patched `kbcli cluster delete-ops --name mssql-bxxmpv-volumeexpansion-tb7k4 --force --auto-approve --namespace default ` OpsRequest mssql-bxxmpv-volumeexpansion-tb7k4 deleted `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-1 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data Success test failover connectionstress check cluster status before cluster-failover-connectionstress check cluster status done cluster_status:Running error: error from server (NotFound): pods "test-db-client-connectionstress-mssql-bxxmpv" not found in namespace "default" `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge pods test-db-client-connectionstress-mssql-bxxmpv --namespace default ` Error from server (NotFound): pods "test-db-client-connectionstress-mssql-bxxmpv" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-connectionstress-mssql-bxxmpv" not found `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. apiVersion: v1 kind: Pod metadata: name: test-db-client-connectionstress-mssql-bxxmpv namespace: default spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/dbclient:test args: - "--host" - "mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local" - "--user" - "sa" - "--password" - "so6N5YwQk4" - "--port" - "1433" - "--database" - "" - "--dbtype" - "sqlserver" - "--test" - "connectionstress" - "--connections" - "10000" - "--duration" - "60" restartPolicy: Never `kubectl apply -f test-db-client-connectionstress-mssql-bxxmpv.yaml` pod/test-db-client-connectionstress-mssql-bxxmpv created apply test-db-client-connectionstress-mssql-bxxmpv.yaml Success `rm -rf test-db-client-connectionstress-mssql-bxxmpv.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-bxxmpv 1/1 Running 0 6s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-bxxmpv 1/1 Running 0 10s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-bxxmpv 1/1 Running 0 15s check pod test-db-client-connectionstress-mssql-bxxmpv status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-bxxmpv 0/1 Completed 0 20s check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql DoNotTerminate Failed Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql cluster_status:Failed check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:22Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:22Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:22Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:65) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:113) at com.apecloud.dbtester.commons.TestExecutor.executeTest(TestExecutor.java:37) at OneClient.executeTest(OneClient.java:108) at OneClient.main(OneClient.java:40) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:65) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:113) at com.apecloud.dbtester.commons.TestExecutor.executeTest(TestExecutor.java:37) at OneClient.executeTest(OneClient.java:108) at OneClient.main(OneClient.java:40) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:65) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:113) at com.apecloud.dbtester.commons.TestExecutor.executeTest(TestExecutor.java:37) at OneClient.executeTest(OneClient.java:108) at OneClient.main(OneClient.java:40) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:65) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:113) at com.apecloud.dbtester.commons.TestExecutor.executeTest(TestExecutor.java:37) at OneClient.executeTest(OneClient.java:108) at OneClient.main(OneClient.java:40) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) ... 4 more Test Result: null Connection Information: Database Type: sqlserver Host: mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local Port: 1433 Database: Table: User: sa Org: Access Mode: mysql Test Type: connectionstress Connection Count: 10000 Duration: 60 seconds `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge pods test-db-client-connectionstress-mssql-bxxmpv --namespace default ` pod/test-db-client-connectionstress-mssql-bxxmpv patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-connectionstress-mssql-bxxmpv" force deleted `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. set db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` set db_client batch data Success check failover pod name failover pod name:mssql-bxxmpv-mssql-2 failover connectionstress Failure `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-bxxmpv-mssql-0 --namespace default -- bash` check readonly db_client batch data Success cluster update terminationPolicy WipeOut `kbcli cluster update mssql-bxxmpv --termination-policy=WipeOut --namespace default ` cluster.apps.kubeblocks.io/mssql-bxxmpv updated check cluster status `kbcli cluster list mssql-bxxmpv --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv default mssql WipeOut Running Sep 12,2025 14:29 UTC+0800 app.kubernetes.io/instance=mssql-bxxmpv,clusterdefinition.kubeblocks.io/name=mssql check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-mssql-0 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:22Gi 192.168.0.70/192.168.0.70 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-1 default mssql-bxxmpv mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:22Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 mssql-bxxmpv-mssql-2 default mssql-bxxmpv mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:22Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:18 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-mssql-2;secondary: mssql-bxxmpv-mssql-0 mssql-bxxmpv-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv` set secret: mssql-bxxmpv-mssql-account-sa `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-mssql-2 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED cluster full backup `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="{.spec.credential.name}"` `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="{.spec.credential.namespace}"` `kubectl get secrets kb-backuprepo-cg5kj -n kb-system -o jsonpath="{.data.accessKeyId}"` `kubectl get secrets kb-backuprepo-cg5kj -n kb-system -o jsonpath="{.data.secretAccessKey}"` KUBEBLOCKS NAMESPACE:kb-system get kubeblocks namespace done `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-system -o jsonpath="{.items[0].data.root-user}"` `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-system -o jsonpath="{.items[0].data.root-password}"` minio_user:kbclitest,minio_password:kbclitest,minio_endpoint:kbcli-test-minio.kb-system.svc.cluster.local:9000 list minio bucket kbcli-test `echo 'mc config host add minioserver http://kbcli-test-minio.kb-system.svc.cluster.local:9000 kbclitest kbclitest;mc ls minioserver' | kubectl exec -it kbcli-test-minio-79497b7755-5x5d8 --namespace kb-system -- bash` Unable to use a TTY - input is not a terminal or the right kind of file list minio bucket done default backuprepo:backuprepo-kbcli-test exists `kbcli cluster backup mssql-bxxmpv --method full --namespace default ` Backup backup-default-mssql-bxxmpv-20250912152943 created successfully, you can view the progress: kbcli cluster list-backups --names=backup-default-mssql-bxxmpv-20250912152943 -n default check backup status `kbcli cluster list-backups mssql-bxxmpv --namespace default ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION DELETION-POLICY CREATE-TIME COMPLETION-TIME EXPIRATION backup-default-mssql-bxxmpv-20250912152943 default mssql-bxxmpv full Running Delete Sep 12,2025 15:29 UTC+0800 backup_status:mssql-bxxmpv-full-Running backup_status:mssql-bxxmpv-full-Running backup_status:mssql-bxxmpv-full-Running check backup status done backup_status:backup-default-mssql-bxxmpv-20250912152943 default mssql-bxxmpv full Completed 41842940 13s Delete Sep 12,2025 15:29 UTC+0800 Sep 12,2025 15:29 UTC+0800 cluster restore backup Error from server (NotFound): opsrequests.operations.kubeblocks.io "mssql-bxxmpv-backup" not found `kbcli cluster describe-backup --names backup-default-mssql-bxxmpv-20250912152943 --namespace default ` Name: backup-default-mssql-bxxmpv-20250912152943 Cluster: mssql-bxxmpv Namespace: default Spec: Method: full Policy Name: mssql-bxxmpv-mssql-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-backup-default-mssql-bxxmpv-20250912152943-540b1446 TargetPodName: mssql-bxxmpv-mssql-2 Phase: Completed Start Time: Sep 12,2025 15:29 UTC+0800 Completion Time: Sep 12,2025 15:29 UTC+0800 Extras: =================== 1 =================== differentialBaseLsn:NULL firstLsn: 42000000016300001 lastLsn: 42000000031000001 name: db1 type: D checkpointLsn: 42000000030700001 databaseBackupLsn: 42000000021600048 =================== 2 =================== checkpointLsn: 48000000288600001 databaseBackupLsn: 42000000155300059 differentialBaseLsn:NULL firstLsn: 42000000145300026 lastLsn: 48000000288900001 name: executions_loop type: D Status: Phase: Completed Total Size: 41842940 ActionSet Name: mssql-full-backup Repository: backuprepo-kbcli-test Duration: 13s Start Time: Sep 12,2025 15:29 UTC+0800 Completion Time: Sep 12,2025 15:29 UTC+0800 Path: /default/mssql-bxxmpv-063f21b8-59ba-4474-aa9e-dcbe9dba91d5/mssql/backup-default-mssql-bxxmpv-20250912152943 Time Range End: Sep 12,2025 15:29 UTC+0800 Warning Events: `kbcli cluster restore mssql-bxxmpv-backup --backup backup-default-mssql-bxxmpv-20250912152943 --namespace default ` Cluster mssql-bxxmpv-backup created check cluster status `kbcli cluster list mssql-bxxmpv-backup --show-labels --namespace default ` NAME NAMESPACE CLUSTER-DEFINITION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-bxxmpv-backup default mssql WipeOut Creating Sep 12,2025 15:30 UTC+0800 clusterdefinition.kubeblocks.io/name=mssql cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-bxxmpv-backup --namespace default ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-bxxmpv-backup-mssql-0 default mssql-bxxmpv-backup mssql Running primary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:22Gi 192.168.0.50/192.168.0.50 Sep 12,2025 15:30 UTC+0800 mssql-bxxmpv-backup-mssql-1 default mssql-bxxmpv-backup mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:22Gi 192.168.0.12/192.168.0.12 Sep 12,2025 15:30 UTC+0800 mssql-bxxmpv-backup-mssql-2 default mssql-bxxmpv-backup mssql Running secondary cn-guangzhou-b 600m / 600m 2254857830400m / 2254857830400m data:22Gi 192.168.0.12/192.168.0.12 Sep 12,2025 15:30 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-bxxmpv-backup-mssql-0;secondary: mssql-bxxmpv-backup-mssql-1 mssql-bxxmpv-backup-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv-backup` set secret: mssql-bxxmpv-backup-mssql-account-sa `kubectl get secrets mssql-bxxmpv-backup-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-backup-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-backup-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-backup-mssql-mssql-ro.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-bxxmpv-backup-mssql-0 --namespace default -- bash` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kbcli cluster describe-backup --names backup-default-mssql-bxxmpv-20250912152943 --namespace default ` Name: backup-default-mssql-bxxmpv-20250912152943 Cluster: mssql-bxxmpv Namespace: default Spec: Method: full Policy Name: mssql-bxxmpv-mssql-backup-policy Actions: dp-backup-0: ActionType: Job WorkloadName: dp-backup-0-backup-default-mssql-bxxmpv-20250912152943-540b1446 TargetPodName: mssql-bxxmpv-mssql-2 Phase: Completed Start Time: Sep 12,2025 15:29 UTC+0800 Completion Time: Sep 12,2025 15:29 UTC+0800 Extras: =================== 1 =================== checkpointLsn: 42000000030700001 databaseBackupLsn: 42000000021600048 differentialBaseLsn:NULL firstLsn: 42000000016300001 lastLsn: 42000000031000001 name: db1 type: D =================== 2 =================== databaseBackupLsn: 42000000155300059 differentialBaseLsn:NULL firstLsn: 42000000145300026 lastLsn: 48000000288900001 name: executions_loop type: D checkpointLsn: 48000000288600001 Status: Phase: Completed Total Size: 41842940 ActionSet Name: mssql-full-backup Repository: backuprepo-kbcli-test Duration: 13s Start Time: Sep 12,2025 15:29 UTC+0800 Completion Time: Sep 12,2025 15:29 UTC+0800 Path: /default/mssql-bxxmpv-063f21b8-59ba-4474-aa9e-dcbe9dba91d5/mssql/backup-default-mssql-bxxmpv-20250912152943 Time Range End: Sep 12,2025 15:29 UTC+0800 Warning Events: cluster connect `kubectl get secrets -l app.kubernetes.io/instance=mssql-bxxmpv-backup` set secret: mssql-bxxmpv-backup-mssql-account-sa `kubectl get secrets mssql-bxxmpv-backup-mssql-account-sa -o jsonpath="{.data.username}"` `kubectl get secrets mssql-bxxmpv-backup-mssql-account-sa -o jsonpath="{.data.password}"` `kubectl get secrets mssql-bxxmpv-backup-mssql-account-sa -o jsonpath="{.data.port}"` DB_USERNAME:sa;DB_PASSWORD:so6N5YwQk4;DB_PORT:1433;DB_DATABASE: No resources found in default namespace. `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-bxxmpv-backup-mssql-mssql-rw.default.svc.cluster.local,1433 -U sa -P 'so6N5YwQk4' -C -Q \"SELECT ag.name AS AvailabilityGroupName,ar.replica_server_name AS ReplicaName,ars.role_desc AS Role,ars.synchronization_health_desc AS SyncHealth FROM sys.availability_groups ag JOIN sys.availability_replicas ar ON ag.group_id = ar.group_id JOIN sys.dm_hadr_availability_replica_states ars ON ar.replica_id = ars.replica_id;\" " | kubectl exec -it mssql-bxxmpv-backup-mssql-0 --namespace default -- bash ` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file AvailabilityGroupName ReplicaName Role SyncHealth -------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------ ------------------------------------------------------------ ag1 mssql-bxxmpv-backup-mssql-0 PRIMARY HEALTHY ag1 mssql-bxxmpv-backup-mssql-1 SECONDARY HEALTHY ag1 mssql-bxxmpv-backup-mssql-2 SECONDARY HEALTHY (3 rows affected) connect cluster Success delete cluster mssql-bxxmpv-backup `kbcli cluster delete mssql-bxxmpv-backup --auto-approve --namespace default ` Cluster mssql-bxxmpv-backup deleted Error from server (NotFound): secrets "mssql-bxxmpv-backup-certificates" not found Error from server (NotFound): secrets "mssql-bxxmpv-backup-certificates" not found Error from server (NotFound): secrets "mssql-bxxmpv-backup-certificates" not found pod_info:mssql-bxxmpv-backup-mssql-0 4/4 Terminating 0 45s No resources found in default namespace. delete cluster pod done No resources found in default namespace. check cluster resource non-exist OK: pvc No resources found in default namespace. delete cluster done No resources found in default namespace. No resources found in default namespace. No resources found in default namespace. cluster delete backup `kubectl patch -p '{"metadata":{"finalizers":[]}}' --type=merge backups backup-default-mssql-bxxmpv-20250912152943 --namespace default ` backup.dataprotection.kubeblocks.io/backup-default-mssql-bxxmpv-20250912152943 patched `kbcli cluster delete-backup mssql-bxxmpv --name backup-default-mssql-bxxmpv-20250912152943 --force --auto-approve --namespace default ` Backup backup-default-mssql-bxxmpv-20250912152943 deleted No opsrequests found in default namespace. cluster list-logs `kbcli cluster list-logs mssql-bxxmpv --namespace default ` No log files found. cluster logs `kbcli cluster logs mssql-bxxmpv --tail 30 --namespace default ` Defaulted container "mssql" out of: mssql, exporter, kbagent, config-manager, init-syncer (init), init-kbagent (init), kbagent-worker (init), install-config-manager-tool (init) == DB ERR == [2025-09-12 07:27:10] 1 packets transmitted, 1 received, 0% packet loss, time 0ms == DB ERR == [2025-09-12 07:27:10] rtt min/avg/max/mdev = 0.118/0.118/0.118/0.000 ms == DB ERR == [2025-09-12 07:27:10] PING mssql-bxxmpv-mssql-1.mssql-bxxmpv-mssql-headless.default.svc.cluster.local (192.168.0.59) 56(84) bytes of data. == DB ERR == [2025-09-12 07:27:10] 64 bytes from 192-168-0-59.mssql-bxxmpv-mssql-mssql-rw.default.svc.cluster.local (192.168.0.59): icmp_seq=1 ttl=64 time=0.022 ms == DB ERR == [2025-09-12 07:27:10] == DB ERR == [2025-09-12 07:27:10] --- mssql-bxxmpv-mssql-1.mssql-bxxmpv-mssql-headless.default.svc.cluster.local ping statistics --- == DB ERR == [2025-09-12 07:27:10] 1 packets transmitted, 1 received, 0% packet loss, time 0ms == DB ERR == [2025-09-12 07:27:10] rtt min/avg/max/mdev = 0.022/0.022/0.022/0.000 ms == DB ERR == [2025-09-12 07:27:10] PING mssql-bxxmpv-mssql-2.mssql-bxxmpv-mssql-headless.default.svc.cluster.local (192.168.0.51) 56(84) bytes of data. == DB ERR == [2025-09-12 07:27:10] 64 bytes from mssql-bxxmpv-mssql-2.mssql-bxxmpv-mssql-headless.default.svc.cluster.local (192.168.0.51): icmp_seq=1 ttl=64 time=0.007 ms == DB ERR == [2025-09-12 07:27:10] == DB ERR == [2025-09-12 07:27:10] --- mssql-bxxmpv-mssql-2.mssql-bxxmpv-mssql-headless.default.svc.cluster.local ping statistics --- == DB ERR == [2025-09-12 07:27:10] 1 packets transmitted, 1 received, 0% packet loss, time 0ms == DB ERR == [2025-09-12 07:27:10] rtt min/avg/max/mdev = 0.007/0.007/0.007/0.000 ms == DB ERR == [2025-09-12 07:27:10] sqlserver init finished, exit code: 0 2025-09-12T07:27:12Z INFO HA wait for db service to be ready 2025-09-12T07:27:12Z INFO HA DB Service is running 2025-09-12T07:27:12Z INFO SQLServer Not all DBs are failover ready {"member": "mssql-bxxmpv-mssql-2", "isAllDBFailoverReady": false} 2025-09-12T07:27:12Z INFO HA DB Service is not healthy, do some recover 2025-09-12T07:27:13Z INFO SQLServer Not all DBs are failover ready {"member": "mssql-bxxmpv-mssql-2", "isAllDBFailoverReady": false} 2025-09-12T07:29:08Z INFO HA Cluster has no leader, attempt to take the leader 2025-09-12T07:29:08Z INFO SQLServer get op timestamp failed {"error": "unable to open tcp connection with host 'mssql-bxxmpv-mssql-1.mssql-bxxmpv-mssql-headless.default.svc:1433': dial tcp 192.168.0.59:1433: connect: connection refused"} 2025-09-12T07:29:08Z INFO SQLServer Replica is now PRIMARY 2025-09-12T07:29:08Z INFO HA Take the leader success! 2025-09-12T07:29:12Z INFO HA This member is Cluster's leader 2025-09-12T07:29:12Z DEBUG HA Refresh leader ttl 2025-09-12T07:30:12Z INFO HA This member is Cluster's leader 2025-09-12T07:30:12Z DEBUG HA Refresh leader ttl 2025-09-12T07:31:12Z INFO HA This member is Cluster's leader 2025-09-12T07:31:12Z DEBUG HA Refresh leader ttl cluster logs specify instance and container `kbcli cluster logs mssql-bxxmpv --tail 30 --instance mssql-bxxmpv-mssql-2 -c config-manager --namespace default ` ts=2025-09-12T07:18:57.781 level=info msg="starting fsnotify VolumeWatcher." volumesDirs=/etc/hscale ts=2025-09-12T07:18:57.781 level=info msg="add watched fs directory: /etc/hscale" ts=2025-09-12T07:18:57.781 level=info msg="fsnotify VolumeWatcher started." ts=2025-09-12T07:18:57.781 level=info logger=grpcProxy msg="container killer is disabled." ts=2025-09-12T07:18:57.781 level=info msg="starting reconfigure service: 0.0.0.0:9901" ts=2025-09-12T07:18:57.781 level=info msg="reconfigure service started." ts=2025-09-12T07:18:57.781 level=info msg="config manager started." delete cluster mssql-bxxmpv `kbcli cluster delete mssql-bxxmpv --auto-approve --namespace default ` Cluster mssql-bxxmpv deleted secret "mssql-bxxmpv-certificates" deleted pod_info:mssql-bxxmpv-mssql-0 4/4 Terminating 0 12m mssql-bxxmpv-mssql-1 4/4 Terminating 0 12m mssql-bxxmpv-mssql-2 4/4 Terminating 2 (4m44s ago) 12m No resources found in default namespace. delete cluster pod done No resources found in default namespace. check cluster resource non-exist OK: pvc No resources found in default namespace. delete cluster done No resources found in default namespace. No resources found in default namespace. No resources found in default namespace. Mssql Test Suite All Done! Test Engine: mssql Test Type: 60 --------------------------------------Mssql (Topology = Replicas 3) Test Result-------------------------------------- [PASSED]|[Create]|[ComponentDefinition=mssql-2022-1.0.1;ComponentVersion=mssql;ServiceVersion=2022.14.0;]|[Description=Create a cluster with the specified component definition mssql-2022-1.0.1 and component version mssql and service version 2022.14.0] [PASSED]|[Connect]|[ComponentName=mssql]|[Description=Connect to the cluster] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [PASSED]|[No-Failover]|[HA=DNS Random;Durations=2m;ComponentName=mssql]|[Description=Simulates conditions where pods experience random IP addresses being returned by the DNS service for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to the DNS service returning random IP addresses.] [PASSED]|[No-Failover]|[HA=DNS Error;Durations=2m;ComponentName=mssql]|[Description=Simulates conditions where pods experience DNS service errors for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to DNS service errors.] [PASSED]|[Failover]|[HA=Network Partition;Durations=2m;ComponentName=mssql]|[Description=Simulates network partition fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to partition network.] [PASSED]|[Failover]|[HA=Network Loss;Durations=2m;ComponentName=mssql]|[Description=Simulates network loss fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to loss network.] [PASSED]|[Failover]|[HA=Pod Failure;Durations=2m;ComponentName=mssql]|[Description=Simulates conditions where pods experience failure for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to failure.] [PASSED]|[VerticalScaling]|[ComponentName=mssql]|[Description=VerticalScaling the cluster specify component mssql] [PASSED]|[No-Failover]|[HA=Network Delay;Durations=2m;ComponentName=mssql]|[Description=Simulates network delay fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to delay network.] [PASSED]|[SwitchOver]|[ComponentName=mssql]|[Description=SwitchOver the cluster specify component mssql] [PASSED]|[Failover]|[HA=Pod Kill;ComponentName=mssql]|[Description=Simulates conditions where pods experience kill for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to kill.] [PASSED]|[Failover]|[HA=Network Bandwidth;Durations=2m;ComponentName=mssql]|[Description=Simulates network bandwidth fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to bandwidth network.] [PASSED]|[Update]|[Monitor=true]|[Description=Update the cluster Monitor enable] [PASSED]|[Failover]|[HA=Network Corrupt;Durations=2m;ComponentName=mssql]|[Description=Simulates network corrupt fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to corrupt network.] [PASSED]|[Stop]|[-]|[Description=Stop the cluster] [PASSED]|[Start]|[-]|[Description=Start the cluster] [PASSED]|[No-Failover]|[HA=Time Offset;Durations=2m;ComponentName=mssql]|[Description=Simulates a time offset scenario thereby testing the application's resilience to potential slowness/unavailability of some replicas due to time offset.] [PASSED]|[No-Failover]|[HA=Full CPU;Durations=2m;ComponentName=mssql]|[Description=Simulates conditions where pods experience CPU full either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high CPU load.] [PASSED]|[No-Failover]|[HA=Network Duplicate;Durations=2m;ComponentName=mssql]|[Description=Simulates network duplicate fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to duplicate network.] [PASSED]|[Failover]|[HA=OOM;Durations=2m;ComponentName=mssql]|[Description=Simulates conditions where pods experience OOM either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Memory load.] [PASSED]|[VolumeExpansion]|[ComponentName=mssql]|[Description=VolumeExpansion the cluster specify component mssql] [PASSED]|[Failover]|[HA=Connection Stress;ComponentName=mssql]|[Description=Simulates conditions where pods experience connection stress either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Connection load.] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Backup]|[BackupMethod=full]|[Description=The cluster full Backup] [PASSED]|[Restore]|[BackupMethod=full]|[Description=The cluster full Restore] [PASSED]|[Connect]|[ComponentName=mssql]|[Description=Connect to the cluster] [PASSED]|[Delete Restore Cluster]|[BackupMethod=full]|[Description=Delete the full restore cluster] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]