source commons files source engines files source kubeblocks files `kubectl get namespace | grep ns-dqbkz ` `kubectl create namespace ns-dqbkz` namespace/ns-dqbkz created create namespace ns-dqbkz done download kbcli `gh release list --repo apecloud/kbcli --limit 100 | (grep "0.9" || true)` `curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v0.9.4-beta.1` Your system is linux_amd64 Installing kbcli ... Downloading ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 32.1M 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 32.1M 100 32.1M 0 0 54.6M 0 --:--:-- --:--:-- --:--:-- 249M kbcli installed successfully. Kubernetes: v1.32.5-eks-5d4a308 KubeBlocks: 0.9.4 kbcli: 0.9.4-beta.1 WARNING: version difference between kbcli (0.9.4-beta.1) and kubeblocks (0.9.4) Make sure your docker service is running and begin your journey with kbcli: kbcli playground init For more information on how to get started, please visit: https://kubeblocks.io download kbcli v0.9.4-beta.1 done Kubernetes: v1.32.5-eks-5d4a308 KubeBlocks: 0.9.4 kbcli: 0.9.4-beta.1 WARNING: version difference between kbcli (0.9.4-beta.1) and kubeblocks (0.9.4) Kubernetes Env: v1.32.5-eks-5d4a308 POD_RESOURCES: No resources found found default storage class: gp3 kubeblocks version is:0.9.4 skip upgrade kubeblocks Error: no repositories to show helm repo add chaos-mesh https://charts.chaos-mesh.org "chaos-mesh" has been added to your repositories add helm chart repo chaos-mesh success chaos mesh already installed check cluster definition set component name:mssql set component version set component version:mssql set service versions:2022.14.0,2022.19.0 set service versions sorted:2022.14.0,2022.19.0 no cluster version found set mssql component definition set mssql component definition mssql-2022 set replicas first:3,2022.14.0|3,2022.19.0 set replicas third:3,2022.19.0 set replicas fourth:3,2022.19.0 set minimum cmpv service version set minimum cmpv service version replicas:3,2022.19.0 REPORT_COUNT:1 CLUSTER_TOPOLOGY: set cluster topology: cluster LIMIT_CPU:0.5 LIMIT_MEMORY:2 storage size: 20 No resources found in ns-dqbkz namespace. termination_policy:WipeOut create 3 replica WipeOut mssql cluster check cluster definition check component definition set component definition by component version check cmpd by labels set component definition1: mssql-2022 by component version:mssql `kubectl apply -f test_create_mssql-yfcvfw.yaml` secret/mssql-yfcvfw-certificates created cluster.apps.kubeblocks.io/mssql-yfcvfw created apply test_create_mssql-yfcvfw.yaml Success `rm -rf test_create_mssql-yfcvfw.yaml` check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Jun 19,2025 18:13 UTC+0800 cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: check pod mssql-yfcvfw-mssql-0 container_name mssql exist password 98Tmq4hsgz check pod mssql-yfcvfw-mssql-0 container_name exporter exist password 98Tmq4hsgz check pod mssql-yfcvfw-mssql-0 container_name lorry exist password 98Tmq4hsgz check pod mssql-yfcvfw-mssql-0 container_name config-manager exist password 98Tmq4hsgz No container logs contain secret password. describe cluster `kbcli cluster describe mssql-yfcvfw --namespace ns-dqbkz ` Name: mssql-yfcvfw Created Time: Jun 19,2025 18:13 UTC+0800 NAMESPACE CLUSTER-DEFINITION VERSION STATUS TERMINATION-POLICY ns-dqbkz mssql Running WipeOut Endpoints: COMPONENT MODE INTERNAL EXTERNAL mssql ReadWrite mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local:1433 mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433 Topology: COMPONENT INSTANCE ROLE STATUS AZ NODE CREATED-TIME mssql mssql-yfcvfw-mssql-0 primary Running us-west-2a ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:13 UTC+0800 mssql mssql-yfcvfw-mssql-1 secondary Running us-west-2a ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql mssql-yfcvfw-mssql-2 secondary Running us-west-2a ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 Resources Allocation: COMPONENT DEDICATED CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS mssql false 500m / 500m 2Gi / 2Gi data:20Gi kb-default-sc Images: COMPONENT TYPE IMAGE mssql docker.io/apecloud/mssql:2022-CU19 Data Protection: BACKUP-REPO AUTO-BACKUP BACKUP-SCHEDULE BACKUP-METHOD BACKUP-RETENTION RECOVERABLE-TIME Show cluster events: kbcli cluster list-events -n ns-dqbkz mssql-yfcvfw `kbcli cluster label mssql-yfcvfw app.kubernetes.io/instance- --namespace ns-dqbkz ` label "app.kubernetes.io/instance" not found. `kbcli cluster label mssql-yfcvfw app.kubernetes.io/instance=mssql-yfcvfw --namespace ns-dqbkz ` `kbcli cluster label mssql-yfcvfw --list --namespace ns-dqbkz ` NAME NAMESPACE LABELS mssql-yfcvfw ns-dqbkz app.kubernetes.io/instance=mssql-yfcvfw clusterdefinition.kubeblocks.io/name=mssql clusterversion.kubeblocks.io/name= label cluster app.kubernetes.io/instance=mssql-yfcvfw Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=mssql-yfcvfw --namespace ns-dqbkz ` `kbcli cluster label mssql-yfcvfw --list --namespace ns-dqbkz ` NAME NAMESPACE LABELS mssql-yfcvfw ns-dqbkz app.kubernetes.io/instance=mssql-yfcvfw case.name=kbcli.test1 clusterdefinition.kubeblocks.io/name=mssql clusterversion.kubeblocks.io/name= label cluster case.name=kbcli.test1 Success `kbcli cluster label mssql-yfcvfw case.name=kbcli.test2 --overwrite --namespace ns-dqbkz ` `kbcli cluster label mssql-yfcvfw --list --namespace ns-dqbkz ` NAME NAMESPACE LABELS mssql-yfcvfw ns-dqbkz app.kubernetes.io/instance=mssql-yfcvfw case.name=kbcli.test2 clusterdefinition.kubeblocks.io/name=mssql clusterversion.kubeblocks.io/name= label cluster case.name=kbcli.test2 Success `kbcli cluster label mssql-yfcvfw case.name- --namespace ns-dqbkz ` `kbcli cluster label mssql-yfcvfw --list --namespace ns-dqbkz ` NAME NAMESPACE LABELS mssql-yfcvfw ns-dqbkz app.kubernetes.io/instance=mssql-yfcvfw clusterdefinition.kubeblocks.io/name=mssql clusterversion.kubeblocks.io/name= delete cluster label case.name Success cluster connect `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"SELECT ag.name AS AvailabilityGroupName,ar.replica_server_name AS ReplicaName,ars.role_desc AS Role,ars.synchronization_health_desc AS SyncHealth FROM sys.availability_groups ag JOIN sys.availability_replicas ar ON ag.group_id = ar.group_id JOIN sys.dm_hadr_availability_replica_states ars ON ar.replica_id = ars.replica_id;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash ` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file AvailabilityGroupName ReplicaName Role SyncHealth -------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------ ------------------------------------------------------------ ag1 mssql-yfcvfw-mssql-0 PRIMARY HEALTHY ag1 mssql-yfcvfw-mssql-1 SECONDARY HEALTHY ag1 mssql-yfcvfw-mssql-2 SECONDARY HEALTHY (3 rows affected) connect cluster Success insert batch data by db client Error from server (NotFound): pods "test-db-client-executionloop-mssql-yfcvfw" not found DB_CLIENT_BATCH_DATA_COUNT: `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-mssql-yfcvfw --namespace ns-dqbkz ` Error from server (NotFound): pods "test-db-client-executionloop-mssql-yfcvfw" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-executionloop-mssql-yfcvfw" not found `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. apiVersion: v1 kind: Pod metadata: name: test-db-client-executionloop-mssql-yfcvfw namespace: ns-dqbkz spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local" - "--user" - "sa" - "--password" - "98Tmq4hsgz" - "--port" - "1433" - "--dbtype" - "sqlserver" - "--test" - "executionloop" - "--duration" - "60" - "--interval" - "1" restartPolicy: Never `kubectl apply -f test-db-client-executionloop-mssql-yfcvfw.yaml` pod/test-db-client-executionloop-mssql-yfcvfw created apply test-db-client-executionloop-mssql-yfcvfw.yaml Success `rm -rf test-db-client-executionloop-mssql-yfcvfw.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 0/1 ContainerCreating 0 6s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 1/1 Running 0 11s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 1/1 Running 0 17s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 1/1 Running 0 23s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 1/1 Running 0 28s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 1/1 Running 0 34s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 1/1 Running 0 40s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 1/1 Running 0 46s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 1/1 Running 0 52s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 1/1 Running 0 58s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 1/1 Running 0 63s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 1/1 Running 0 69s check pod test-db-client-executionloop-mssql-yfcvfw status done pod_status:NAME READY STATUS RESTARTS AGE test-db-client-executionloop-mssql-yfcvfw 0/1 Completed 0 75s check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Running Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED --host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local --user sa --password 98Tmq4hsgz --port 1433 --dbtype sqlserver --test executionloop --duration 60 --interval 1 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider] 10:16:48.133 [main] DEBUG com.clickhouse.jdbc.ClickHouseDriver -- ClickHouse Driver 0.0.0.0(JDBC: 0.0.0.0) registered Execution loop start: create databases executions_loop IF NOT EXISTS (SELECT * FROM sys.databases WHERE name = 'executions_loop') CREATE DATABASE executions_loop drop table executions_loop_table IF OBJECT_ID('executions_loop..executions_loop_table', 'U') IS NOT NULL DROP TABLE executions_loop..executions_loop_table create table executions_loop_table Execution loop start: INSERT INTO executions_loop..executions_loop_table (value, tinyint_col, smallint_col, int_col, bigint_col, float_col, double_col, decimal_col, date_col, time_col, datetime_col, timestamp_col, year_col, char_col, text_col, blob_col, binary_col, varbinary_col, enum_col, set_col) VALUES ('executions_loop_test_1', 11, 15730, 1364176940, -7352019272233569146, 0.36827451843757386, 0.9062307658881418, 47.11698501942519, '2025-06-19', '10:16:54', '2025-06-19 10:16:54.37', SYSUTCDATETIME(), '2014', 'ijRwZjmmMQ', 'T0j3AxEScPyXxkBqTgf0K43LH3Ky8poLvupDy72qGsGhqkPvAQeNoJduEdEJvZEVPRpTxKloc2ocaFVL8CKCokT49xWgtt5JZuzcSR06UhcObBqj0raZz2GoDaqukjzrkkz1LG9mxF4LAN1yKvjZq91Aaly6ZbZjnOPQhYnRanVSJ5WBDdHw1JJmltm9sRizWPPNmT4ovvMCyoV6spvQINDdbBHQj8bHeUarm30zFYR4OCTX9iByJp3ywynhROl', 0xd9043e3109946915a2e5, 0x2d9c933150ba41056ec5, 0x5e20b0d10bf4d52d6cb43a96989801b731a8769270aa3fc69ead9502a987f08ec5e6079a8fa7c4d63f29f2cab75a64b321b5dd311d6c5a1ec4d25b94f56897bef36f97001331749a301c5ee114481fec307db6b2b2bfba3bc9cf0ecce68d4438de1625e6c7451b1c0e89c3034588e289306fde3b30aa38aee748f133320fb016d381ec689e25ffd41e99f47540d540ec8626618e9b23d2970ef3732832da20d3f038357cceafc8a8995df513e3e36183ade951567ef3ec1e5e765bf066210d65a98cb53a5487b7522b5a8d696d5c6949b178e674649d476fb020d09dee5c28be46ccfbc26e47650376df8fa5036394c7b52bad8c605d6b2a0e68c11d11db00, 'Option2', 'Value1') [ 1s ] executions total: 1 successful: 1 failed: 0 disconnect: 0 [ 2s ] executions total: 118 successful: 118 failed: 0 disconnect: 0 [ 3s ] executions total: 179 successful: 179 failed: 0 disconnect: 0 [ 4s ] executions total: 228 successful: 228 failed: 0 disconnect: 0 Failed to execute query: com.microsoft.sqlserver.jdbc.SQLServerException: Unable to access database 'executions_loop' because it lacks a quorum of nodes for high availability. Try the operation again later. [2025-06-19 10:16:58.259] Connection error occurred! Failed to execute query: com.microsoft.sqlserver.jdbc.SQLServerException: Unable to access database 'executions_loop' because it lacks a quorum of nodes for high availability. Try the operation again later. [ 5s ] executions total: 291 successful: 289 failed: 2 disconnect: 1 Failed to execute query: com.microsoft.sqlserver.jdbc.SQLServerException: Unable to access database 'executions_loop' because it lacks a quorum of nodes for high availability. Try the operation again later. [ 6s ] executions total: 292 successful: 289 failed: 3 disconnect: 1 [2025-06-19 10:16:58.259] Connection error occurred! [2025-06-19 10:17:02.065] Connection successfully recovered! The connection was restored in 3806 milliseconds. [ 7s ] executions total: 293 successful: 290 failed: 3 disconnect: 1 [ 8s ] executions total: 324 successful: 321 failed: 3 disconnect: 1 [ 9s ] executions total: 365 successful: 362 failed: 3 disconnect: 1 [ 10s ] executions total: 447 successful: 444 failed: 3 disconnect: 1 [ 11s ] executions total: 524 successful: 521 failed: 3 disconnect: 1 [ 12s ] executions total: 602 successful: 599 failed: 3 disconnect: 1 [ 13s ] executions total: 673 successful: 670 failed: 3 disconnect: 1 [ 14s ] executions total: 712 successful: 709 failed: 3 disconnect: 1 [ 15s ] executions total: 784 successful: 781 failed: 3 disconnect: 1 [ 16s ] executions total: 865 successful: 862 failed: 3 disconnect: 1 [ 17s ] executions total: 928 successful: 925 failed: 3 disconnect: 1 [ 18s ] executions total: 998 successful: 995 failed: 3 disconnect: 1 [ 19s ] executions total: 1069 successful: 1066 failed: 3 disconnect: 1 [ 20s ] executions total: 1129 successful: 1126 failed: 3 disconnect: 1 [ 21s ] executions total: 1198 successful: 1195 failed: 3 disconnect: 1 [ 22s ] executions total: 1282 successful: 1279 failed: 3 disconnect: 1 [ 23s ] executions total: 1322 successful: 1319 failed: 3 disconnect: 1 [ 24s ] executions total: 1391 successful: 1388 failed: 3 disconnect: 1 [ 25s ] executions total: 1455 successful: 1452 failed: 3 disconnect: 1 [ 26s ] executions total: 1483 successful: 1480 failed: 3 disconnect: 1 [ 27s ] executions total: 1560 successful: 1557 failed: 3 disconnect: 1 [ 28s ] executions total: 1646 successful: 1643 failed: 3 disconnect: 1 [ 29s ] executions total: 1719 successful: 1716 failed: 3 disconnect: 1 [ 30s ] executions total: 1810 successful: 1807 failed: 3 disconnect: 1 [ 31s ] executions total: 1878 successful: 1875 failed: 3 disconnect: 1 [ 32s ] executions total: 1938 successful: 1935 failed: 3 disconnect: 1 [ 33s ] executions total: 2016 successful: 2013 failed: 3 disconnect: 1 [ 34s ] executions total: 2094 successful: 2091 failed: 3 disconnect: 1 [ 35s ] executions total: 2141 successful: 2138 failed: 3 disconnect: 1 [ 36s ] executions total: 2220 successful: 2217 failed: 3 disconnect: 1 [ 37s ] executions total: 2299 successful: 2296 failed: 3 disconnect: 1 [ 38s ] executions total: 2353 successful: 2350 failed: 3 disconnect: 1 [ 39s ] executions total: 2430 successful: 2427 failed: 3 disconnect: 1 [ 40s ] executions total: 2491 successful: 2488 failed: 3 disconnect: 1 [ 41s ] executions total: 2556 successful: 2553 failed: 3 disconnect: 1 [ 42s ] executions total: 2652 successful: 2649 failed: 3 disconnect: 1 [ 43s ] executions total: 2695 successful: 2692 failed: 3 disconnect: 1 [ 44s ] executions total: 2722 successful: 2719 failed: 3 disconnect: 1 [ 45s ] executions total: 2802 successful: 2799 failed: 3 disconnect: 1 [ 46s ] executions total: 2890 successful: 2887 failed: 3 disconnect: 1 [ 47s ] executions total: 2966 successful: 2963 failed: 3 disconnect: 1 [ 48s ] executions total: 3047 successful: 3044 failed: 3 disconnect: 1 [ 49s ] executions total: 3105 successful: 3102 failed: 3 disconnect: 1 [ 50s ] executions total: 3182 successful: 3179 failed: 3 disconnect: 1 [ 51s ] executions total: 3266 successful: 3263 failed: 3 disconnect: 1 [ 52s ] executions total: 3310 successful: 3307 failed: 3 disconnect: 1 [ 53s ] executions total: 3400 successful: 3397 failed: 3 disconnect: 1 [ 60s ] executions total: 3404 successful: 3401 failed: 3 disconnect: 1 Test Result: Total Executions: 3404 Successful Executions: 3401 Failed Executions: 3 Disconnection Counts: 1 Connection Information: Database Type: sqlserver Host: mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local Port: 1433 Database: Table: User: sa Org: Access Mode: mysql Test Type: executionloop Query: Duration: 60 seconds Interval: 1 seconds DB_CLIENT_BATCH_DATA_COUNT: 3401 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-executionloop-mssql-yfcvfw --namespace ns-dqbkz ` pod/test-db-client-executionloop-mssql-yfcvfw patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-executionloop-mssql-yfcvfw" force deleted test failover dnsrandom check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mssql-yfcvfw" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnsrandom-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-0 mode: all action: random duration: 2m `kubectl apply -f test-chaos-mesh-dnsrandom-mssql-yfcvfw.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnsrandom-mssql-yfcvfw created apply test-chaos-mesh-dnsrandom-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-dnsrandom-mssql-yfcvfw.yaml` dnsrandom chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Running Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnsrandom-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mssql-yfcvfw" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnsrandom-mssql-yfcvfw" not found check failover pod name failover pod name:mssql-yfcvfw-mssql-0 failover dnsrandom Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success test failover networkpartition check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-mssql-yfcvfw" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkpartition-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-0 action: partition mode: all target: mode: all selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-1 direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkpartition-mssql-yfcvfw.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkpartition-mssql-yfcvfw created apply test-chaos-mesh-networkpartition-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-networkpartition-mssql-yfcvfw.yaml` networkpartition chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Running Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-2;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkpartition-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkpartition-mssql-yfcvfw" force deleted networkchaos.chaos-mesh.org/test-chaos-mesh-networkpartition-mssql-yfcvfw patched check failover pod name failover pod name:mssql-yfcvfw-mssql-2 failover networkpartition Failure test failover networkbandwidthover check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mssql-yfcvfw" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkbandwidthover-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-2 action: bandwidth mode: all bandwidth: rate: '1bps' limit: 20971520 buffer: 10000 duration: 2m `kubectl apply -f test-chaos-mesh-networkbandwidthover-mssql-yfcvfw.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkbandwidthover-mssql-yfcvfw created apply test-chaos-mesh-networkbandwidthover-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-networkbandwidthover-mssql-yfcvfw.yaml` networkbandwidthover chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-1;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkbandwidthover-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mssql-yfcvfw" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkbandwidthover-mssql-yfcvfw" not found check failover pod name failover pod name:mssql-yfcvfw-mssql-1 failover networkbandwidthover Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success test failover networkdelay check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mssql-yfcvfw" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkdelay-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-1 mode: all action: delay delay: latency: 2000ms correlation: '100' jitter: 0ms direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkdelay-mssql-yfcvfw.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkdelay-mssql-yfcvfw created apply test-chaos-mesh-networkdelay-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-networkdelay-mssql-yfcvfw.yaml` networkdelay chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-1;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkdelay-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mssql-yfcvfw" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkdelay-mssql-yfcvfw" not found check failover pod name failover pod name:mssql-yfcvfw-mssql-1 failover networkdelay Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success test failover dnserror check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mssql-yfcvfw" not found Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: DNSChaos metadata: name: test-chaos-mesh-dnserror-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-1 mode: all action: error duration: 2m `kubectl apply -f test-chaos-mesh-dnserror-mssql-yfcvfw.yaml` dnschaos.chaos-mesh.org/test-chaos-mesh-dnserror-mssql-yfcvfw created apply test-chaos-mesh-dnserror-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-dnserror-mssql-yfcvfw.yaml` dnserror chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Running Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-1;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge DNSChaos test-chaos-mesh-dnserror-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mssql-yfcvfw" force deleted Error from server (NotFound): dnschaos.chaos-mesh.org "test-chaos-mesh-dnserror-mssql-yfcvfw" not found check failover pod name failover pod name:mssql-yfcvfw-mssql-1 failover dnserror Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success test switchover cluster promote check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster promote mssql-yfcvfw --auto-approve --force=true --component mssql --instance mssql-yfcvfw-mssql-0 --namespace ns-dqbkz ` component:mssql OpsRequest mssql-yfcvfw-switchover-vn5p8 created successfully, you can view the progress: kbcli cluster describe-ops mssql-yfcvfw-switchover-vn5p8 -n ns-dqbkz check ops status `kbcli cluster list-ops mssql-yfcvfw --status all --namespace ns-dqbkz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Running Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-1;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED check ops status `kbcli cluster list-ops mssql-yfcvfw --status all --namespace ns-dqbkz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Running 0/1 Jun 19,2025 18:32 UTC+0800 check ops status done ops_status:mssql-yfcvfw-switchover-vn5p8 ns-dqbkz Switchover mssql-yfcvfw mssql Succeed 1/1 Jun 19,2025 18:32 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mssql-yfcvfw-switchover-vn5p8 --namespace ns-dqbkz ` opsrequest.apps.kubeblocks.io/mssql-yfcvfw-switchover-vn5p8 patched `kbcli cluster delete-ops --name mssql-yfcvfw-switchover-vn5p8 --force --auto-approve --namespace ns-dqbkz ` OpsRequest mssql-yfcvfw-switchover-vn5p8 deleted `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success switchover pod:mssql-yfcvfw-mssql-0 switchover success test failover networkcorruptover check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mssql-yfcvfw" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkcorruptover-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-0 mode: all action: corrupt corrupt: corrupt: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkcorruptover-mssql-yfcvfw.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkcorruptover-mssql-yfcvfw created apply test-chaos-mesh-networkcorruptover-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-networkcorruptover-mssql-yfcvfw.yaml` networkcorruptover chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-2;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkcorruptover-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mssql-yfcvfw" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkcorruptover-mssql-yfcvfw" not found check failover pod name failover pod name:mssql-yfcvfw-mssql-0 failover networkcorruptover Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success test failover delete pod:mssql-yfcvfw-mssql-0 `kubectl delete pod mssql-yfcvfw-mssql-0 --namespace ns-dqbkz ` pod "mssql-yfcvfw-mssql-0" deleted check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:41 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED check failover pod name failover pod name:mssql-yfcvfw-mssql-0 checking failover... `kubectl delete pod mssql-yfcvfw-mssql-0 --namespace ns-dqbkz ` pod "mssql-yfcvfw-mssql-0" deleted check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:43 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED failover pod name:mssql-yfcvfw-mssql-0 checking failover... `kubectl delete pod mssql-yfcvfw-mssql-0 --namespace ns-dqbkz ` pod "mssql-yfcvfw-mssql-0" deleted check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:45 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role No resources found in ns-dqbkz namespace. primary: mssql-yfcvfw-mssql-0;secondary: No resources found in ns-dqbkz namespace. primary: mssql-yfcvfw-mssql-0;secondary: check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED failover pod name:mssql-yfcvfw-mssql-0 checking failover... `kubectl delete pod mssql-yfcvfw-mssql-0 --namespace ns-dqbkz ` pod "mssql-yfcvfw-mssql-0" deleted check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating [Error] check cluster status timeout --------------------------------------get cluster mssql-yfcvfw yaml-------------------------------------- `kubectl get cluster mssql-yfcvfw -o yaml --namespace ns-dqbkz ` apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: annotations: kubeblocks.io/extra-env: '***"MSSQL_PID":"Enterprise","MSSQL_COLLATION":"Chinese_PRC_CI_AS"***' kubeblocks.io/reconcile: "2025-06-19T10:37:45.992031233Z" kubectl.kubernetes.io/last-applied-configuration: | ***"apiVersion":"apps.kubeblocks.io/v1alpha1","kind":"Cluster","metadata":***"annotations":***"kubeblocks.io/extra-env":"***\"MSSQL_PID\":\"Enterprise\",\"MSSQL_COLLATION\":\"Chinese_PRC_CI_AS\"***"***,"name":"mssql-yfcvfw","namespace":"ns-dqbkz"***,"spec":***"clusterDefinitionRef":"mssql","componentSpecs":[***"env":[***"name":"MSSQL_PRIVATE_ENCRYPTION_PASSWORD","valueFrom":***"secretKeyRef":***"key":"private_key_password","name":"mssql-yfcvfw-certificates"***,***"name":"HOST_NETWORK","value":"false"***,***"name":"DEFAULT_DB_NAME","value":"db1"***,***"name":"IS_REMOTE_STANDBY","value":"false"***,***"name":"REMOTE_SESSION_TIMEOUT_SECONDS","value":"30"***],"name":"mssql","replicas":3,"resources":***"limits":***"cpu":"500m","memory":"2Gi"***,"requests":***"cpu":"500m","memory":"2Gi"***,"serviceVersion":"2022.19.0","volumeClaimTemplates":[***"name":"data","spec":***"accessModes":["ReadWriteOnce"],"resources":***"requests":***"storage":"20Gi"***,"storageClassName":null***],"volumes":[***"name":"certificates","secret":***"secretName":"mssql-yfcvfw-certificates"***]***],"terminationPolicy":"WipeOut","topology":"cluster"*** creationTimestamp: "2025-06-19T10:13:18Z" finalizers: - cluster.kubeblocks.io/finalizer generation: 3 labels: app.kubernetes.io/instance: mssql-yfcvfw clusterdefinition.kubeblocks.io/name: mssql clusterversion.kubeblocks.io/name: "" name: mssql-yfcvfw namespace: ns-dqbkz resourceVersion: "48424" uid: 527fd339-2f94-4750-8eb8-25bb89e4f787 spec: clusterDefinitionRef: mssql componentSpecs: - componentDef: mssql-2022 env: - name: MSSQL_PRIVATE_ENCRYPTION_PASSWORD valueFrom: secretKeyRef: key: private_key_password name: mssql-yfcvfw-certificates - name: HOST_NETWORK value: "false" - name: DEFAULT_DB_NAME value: db1 - name: IS_REMOTE_STANDBY value: "false" - name: REMOTE_SESSION_TIMEOUT_SECONDS value: "30" name: mssql replicas: 3 resources: limits: cpu: 500m memory: 2Gi requests: cpu: 500m memory: 2Gi serviceVersion: 2022.19.0 volumeClaimTemplates: - name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi volumes: - name: certificates secret: secretName: mssql-yfcvfw-certificates resources: cpu: "0" memory: "0" storage: size: "0" terminationPolicy: WipeOut topology: cluster status: clusterDefGeneration: 1 components: mssql: message: InstanceSet/mssql-yfcvfw-mssql: Role probe timeout, check whether the application is available phase: Updating podsReady: false podsReadyTime: "2025-06-19T10:48:11Z" conditions: - lastTransitionTime: "2025-06-19T10:13:19Z" message: 'The operator has started the provisioning of Cluster: mssql-yfcvfw' observedGeneration: 3 reason: PreCheckSucceed status: "True" type: ProvisioningStarted - lastTransitionTime: "2025-06-19T10:13:19Z" message: Successfully applied for resources observedGeneration: 3 reason: ApplyResourcesSucceed status: "True" type: ApplyResources - lastTransitionTime: "2025-06-19T10:48:13Z" message: 'pods are not ready in Components: [mssql], refer to related component message in Cluster.status.components' reason: ReplicasNotReady status: "False" type: ReplicasReady - lastTransitionTime: "2025-06-19T10:48:13Z" message: 'pods are unavailable in Components: [mssql], refer to related component message in Cluster.status.components' reason: ComponentsNotReady status: "False" type: Ready observedGeneration: 3 phase: Updating ------------------------------------------------------------------------------------------------------------------ --------------------------------------describe cluster mssql-yfcvfw-------------------------------------- `kubectl describe cluster mssql-yfcvfw --namespace ns-dqbkz ` Name: mssql-yfcvfw Namespace: ns-dqbkz Labels: app.kubernetes.io/instance=mssql-yfcvfw clusterdefinition.kubeblocks.io/name=mssql clusterversion.kubeblocks.io/name= Annotations: kubeblocks.io/extra-env: ***"MSSQL_PID":"Enterprise","MSSQL_COLLATION":"Chinese_PRC_CI_AS"*** kubeblocks.io/reconcile: 2025-06-19T10:37:45.992031233Z API Version: apps.kubeblocks.io/v1alpha1 Kind: Cluster Metadata: Creation Timestamp: 2025-06-19T10:13:18Z Finalizers: cluster.kubeblocks.io/finalizer Generation: 3 Resource Version: 48424 UID: 527fd339-2f94-4750-8eb8-25bb89e4f787 Spec: Cluster Definition Ref: mssql Component Specs: Component Def: mssql-2022 Env: Name: MSSQL_PRIVATE_ENCRYPTION_PASSWORD Value From: Secret Key Ref: Key: private_key_password Name: mssql-yfcvfw-certificates Name: HOST_NETWORK Value: false Name: DEFAULT_DB_NAME Value: db1 Name: IS_REMOTE_STANDBY Value: false Name: REMOTE_SESSION_TIMEOUT_SECONDS Value: 30 Name: mssql Replicas: 3 Resources: Limits: Cpu: 500m Memory: 2Gi Requests: Cpu: 500m Memory: 2Gi Service Version: 2022.19.0 Volume Claim Templates: Name: data Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 20Gi Volumes: Name: certificates Secret: Secret Name: mssql-yfcvfw-certificates Resources: Cpu: 0 Memory: 0 Storage: Size: 0 Termination Policy: WipeOut Topology: cluster Status: Cluster Def Generation: 1 Components: Mssql: Message: InstanceSet/mssql-yfcvfw-mssql: Role probe timeout, check whether the application is available Phase: Updating Pods Ready: false Pods Ready Time: 2025-06-19T10:48:11Z Conditions: Last Transition Time: 2025-06-19T10:13:19Z Message: The operator has started the provisioning of Cluster: mssql-yfcvfw Observed Generation: 3 Reason: PreCheckSucceed Status: True Type: ProvisioningStarted Last Transition Time: 2025-06-19T10:13:19Z Message: Successfully applied for resources Observed Generation: 3 Reason: ApplyResourcesSucceed Status: True Type: ApplyResources Last Transition Time: 2025-06-19T10:48:13Z Message: pods are not ready in Components: [mssql], refer to related component message in Cluster.status.components Reason: ReplicasNotReady Status: False Type: ReplicasReady Last Transition Time: 2025-06-19T10:48:13Z Message: pods are unavailable in Components: [mssql], refer to related component message in Cluster.status.components Reason: ComponentsNotReady Status: False Type: Ready Observed Generation: 3 Phase: Updating Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal PreCheckSucceed 42m (x2 over 42m) cluster-controller The operator has started the provisioning of Cluster: mssql-yfcvfw Normal ApplyResourcesSucceed 42m (x2 over 42m) cluster-controller Successfully applied for resources Normal ComponentPhaseTransition 42m cluster-controller component is Creating Warning Abnormal 34m (x2 over 34m) cluster-controller Cluster: mssql-yfcvfw is Abnormal, check according to the components message Warning ComponentsNotReady 34m (x2 over 34m) cluster-controller pods are unavailable in Components: [mssql], refer to related component message in Cluster.status.components Warning ReplicasNotReady 34m (x2 over 34m) cluster-controller pods are not ready in Components: [mssql], refer to related component message in Cluster.status.components Normal Running 33m (x3 over 40m) cluster-controller Cluster: mssql-yfcvfw is ready, current phase is Running Normal ClusterReady 33m (x3 over 40m) cluster-controller Cluster: mssql-yfcvfw is ready, current phase is Running Normal AllReplicasReady 33m (x3 over 40m) cluster-controller all pods of components are ready, waiting for the probe detection successful Warning Unhealthy 30m (x16 over 32m) event-controller Pod mssql-yfcvfw-mssql-2: Readiness probe failed: Get "http://172.31.10.115:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Warning Unhealthy 27m (x25 over 29m) event-controller Pod mssql-yfcvfw-mssql-1: Readiness probe failed: Get "http://172.31.4.184:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Normal ComponentPhaseTransition 21m (x3 over 34m) cluster-controller component is Abnormal Warning Unhealthy 19m event-controller Pod mssql-yfcvfw-mssql-0: Readiness probe failed: Get "http://172.31.6.122:3501/v1.0/checkrole": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Normal ComponentPhaseTransition 15m (x8 over 40m) cluster-controller component is Running Normal ComponentPhaseTransition 11m (x6 over 32m) cluster-controller component is Updating ------------------------------------------------------------------------------------------------------------------ check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:47 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get pod name timeout failover pod name: `kubectl delete pod mssql-yfcvfw-mssql-0 --namespace ns-dqbkz ` pod "mssql-yfcvfw-mssql-0" deleted check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:56 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED failover pod name:mssql-yfcvfw-mssql-0 checking failover... `kubectl delete pod mssql-yfcvfw-mssql-0 --namespace ns-dqbkz ` pod "mssql-yfcvfw-mssql-0" deleted check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 18:58 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 18:13 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 18:13 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED check failover pod name timeout cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart mssql-yfcvfw --auto-approve --force=true --namespace ns-dqbkz ` OpsRequest mssql-yfcvfw-restart-sxx9q created successfully, you can view the progress: kbcli cluster describe-ops mssql-yfcvfw-restart-sxx9q -n ns-dqbkz check ops status `kbcli cluster list-ops mssql-yfcvfw --status all --namespace ns-dqbkz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-yfcvfw-restart-sxx9q ns-dqbkz Restart mssql-yfcvfw mssql Running 0/3 Jun 19,2025 19:00 UTC+0800 check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:02 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:01 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:00 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED check ops status `kbcli cluster list-ops mssql-yfcvfw --status all --namespace ns-dqbkz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-yfcvfw-restart-sxx9q ns-dqbkz Restart mssql-yfcvfw mssql Succeed 3/3 Jun 19,2025 19:00 UTC+0800 check ops status done ops_status:mssql-yfcvfw-restart-sxx9q ns-dqbkz Restart mssql-yfcvfw mssql Succeed 3/3 Jun 19,2025 19:00 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mssql-yfcvfw-restart-sxx9q --namespace ns-dqbkz ` opsrequest.apps.kubeblocks.io/mssql-yfcvfw-restart-sxx9q patched `kbcli cluster delete-ops --name mssql-yfcvfw-restart-sxx9q --force --auto-approve --namespace ns-dqbkz ` OpsRequest mssql-yfcvfw-restart-sxx9q deleted `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` test failover connectionstress check node drain check node drain success Error from server (NotFound): pods "test-db-client-connectionstress-mssql-yfcvfw" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstress-mssql-yfcvfw --namespace ns-dqbkz ` Error from server (NotFound): pods "test-db-client-connectionstress-mssql-yfcvfw" not found Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): pods "test-db-client-connectionstress-mssql-yfcvfw" not found `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. apiVersion: v1 kind: Pod metadata: name: test-db-client-connectionstress-mssql-yfcvfw namespace: ns-dqbkz spec: containers: - name: test-dbclient imagePullPolicy: IfNotPresent image: docker.io/apecloud/dbclient:test args: - "--host" - "mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local" - "--user" - "sa" - "--password" - "98Tmq4hsgz" - "--port" - "1433" - "--database" - "" - "--dbtype" - "sqlserver" - "--test" - "connectionstress" - "--connections" - "10000" - "--duration" - "60" restartPolicy: Never `kubectl apply -f test-db-client-connectionstress-mssql-yfcvfw.yaml` pod/test-db-client-connectionstress-mssql-yfcvfw created apply test-db-client-connectionstress-mssql-yfcvfw.yaml Success `rm -rf test-db-client-connectionstress-mssql-yfcvfw.yaml` check pod status pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 5s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 10s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 16s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 22s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 28s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 34s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 39s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 45s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 51s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 57s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 63s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 68s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 74s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 80s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 86s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 92s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 97s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 103s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 109s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 115s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 2m pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 2m6s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 2m12s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 2m18s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 2m24s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 2m29s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 2m35s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 2m41s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 2m47s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 2m52s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 2m58s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 3m4s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 3m10s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 3m16s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 3m21s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 3m27s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 3m33s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 3m39s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 3m44s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 3m50s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 3m56s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 4m2s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 4m8s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 4m13s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 4m19s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 4m25s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 4m31s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 4m36s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 4m42s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 4m48s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 4m54s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 5m pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 5m5s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 5m11s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 5m17s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 5m23s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 5m28s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 5m34s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 5m40s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 5m46s pod_status:NAME READY STATUS RESTARTS AGE test-db-client-connectionstress-mssql-yfcvfw 1/1 Running 0 5m52s check pod test-db-client-connectionstress-mssql-yfcvfw status timeout --------------------------------------get pod test-db-client-connectionstress-mssql-yfcvfw yaml-------------------------------------- `kubectl get pod test-db-client-connectionstress-mssql-yfcvfw -o yaml --namespace ns-dqbkz ` apiVersion: v1 kind: Pod metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | ***"apiVersion":"v1","kind":"Pod","metadata":***"annotations":***,"name":"test-db-client-connectionstress-mssql-yfcvfw","namespace":"ns-dqbkz"***,"spec":***"containers":[***"args":["--host","mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local","--user","sa","--password","98Tmq4hsgz","--port","1433","--database","","--dbtype","sqlserver","--test","connectionstress","--connections","10000","--duration","60"],"image":"docker.io/apecloud/dbclient:test","imagePullPolicy":"IfNotPresent","name":"test-dbclient"***],"restartPolicy":"Never"*** creationTimestamp: "2025-06-19T11:04:37Z" name: test-db-client-connectionstress-mssql-yfcvfw namespace: ns-dqbkz resourceVersion: "56796" uid: dca143ec-7437-4ece-8f5a-77ff56ea21b2 spec: containers: - args: - --host - mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local - --user - sa - --password - 98Tmq4hsgz - --port - "1433" - --database - "" - --dbtype - sqlserver - --test - connectionstress - --connections - "10000" - --duration - "60" image: docker.io/apecloud/dbclient:test imagePullPolicy: IfNotPresent name: test-dbclient resources: *** terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: kube-api-access-rg8v2 readOnly: true dnsPolicy: ClusterFirst enableServiceLinks: true nodeName: ip-172-31-3-103.us-west-2.compute.internal preemptionPolicy: PreemptLowerPriority priority: 0 restartPolicy: Never schedulerName: default-scheduler securityContext: *** serviceAccount: default serviceAccountName: default terminationGracePeriodSeconds: 30 tolerations: - effect: NoExecute key: node.kubernetes.io/not-ready operator: Exists tolerationSeconds: 300 - effect: NoExecute key: node.kubernetes.io/unreachable operator: Exists tolerationSeconds: 300 volumes: - name: kube-api-access-rg8v2 projected: defaultMode: 420 sources: - serviceAccountToken: expirationSeconds: 3607 path: token - configMap: items: - key: ca.crt path: ca.crt name: kube-root-ca.crt - downwardAPI: items: - fieldRef: apiVersion: v1 fieldPath: metadata.namespace path: namespace status: conditions: - lastProbeTime: null lastTransitionTime: "2025-06-19T11:04:38Z" status: "True" type: PodReadyToStartContainers - lastProbeTime: null lastTransitionTime: "2025-06-19T11:04:37Z" status: "True" type: Initialized - lastProbeTime: null lastTransitionTime: "2025-06-19T11:04:38Z" status: "True" type: Ready - lastProbeTime: null lastTransitionTime: "2025-06-19T11:04:38Z" status: "True" type: ContainersReady - lastProbeTime: null lastTransitionTime: "2025-06-19T11:04:37Z" status: "True" type: PodScheduled containerStatuses: - containerID: containerd://04eb748aeeb23d41f09e118de3f50ab53429f4697b113fee4c926536976bcbeb image: docker.io/apecloud/dbclient:test imageID: docker.io/apecloud/dbclient@sha256:94dcfdb6d1e9c728bba32050d67228b14611badb037c61c1740ebfc68677d43f lastState: *** name: test-dbclient ready: true restartCount: 0 started: true state: running: startedAt: "2025-06-19T11:04:37Z" volumeMounts: - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: kube-api-access-rg8v2 readOnly: true recursiveReadOnly: Disabled hostIP: 172.31.3.103 hostIPs: - ip: 172.31.3.103 phase: Running podIP: 172.31.4.130 podIPs: - ip: 172.31.4.130 qosClass: BestEffort startTime: "2025-06-19T11:04:37Z" ------------------------------------------------------------------------------------------------------------------ --------------------------------------describe pod test-db-client-connectionstress-mssql-yfcvfw-------------------------------------- `kubectl describe pod test-db-client-connectionstress-mssql-yfcvfw --namespace ns-dqbkz ` Name: test-db-client-connectionstress-mssql-yfcvfw Namespace: ns-dqbkz Priority: 0 Service Account: default Node: ip-172-31-3-103.us-west-2.compute.internal/172.31.3.103 Start Time: Thu, 19 Jun 2025 19:04:37 +0800 Labels: Annotations: Status: Running IP: 172.31.4.130 IPs: IP: 172.31.4.130 Containers: test-dbclient: Container ID: containerd://04eb748aeeb23d41f09e118de3f50ab53429f4697b113fee4c926536976bcbeb Image: docker.io/apecloud/dbclient:test Image ID: docker.io/apecloud/dbclient@sha256:94dcfdb6d1e9c728bba32050d67228b14611badb037c61c1740ebfc68677d43f Port: Host Port: Args: --host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local --user sa --password 98Tmq4hsgz --port 1433 --database --dbtype sqlserver --test connectionstress --connections 10000 --duration 60 State: Running Started: Thu, 19 Jun 2025 19:04:37 +0800 Ready: True Restart Count: 0 Environment: Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-rg8v2 (ro) Conditions: Type Status PodReadyToStartContainers True Initialized True Ready True ContainersReady True PodScheduled True Volumes: kube-api-access-rg8v2: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: BestEffort Node-Selectors: Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 5m54s default-scheduler Successfully assigned ns-dqbkz/test-db-client-connectionstress-mssql-yfcvfw to ip-172-31-3-103.us-west-2.compute.internal Normal Pulled 5m54s kubelet Container image "docker.io/apecloud/dbclient:test" already present on machine Normal Created 5m54s kubelet Created container: test-dbclient Normal Started 5m54s kubelet Started container test-dbclient ------------------------------------------------------------------------------------------------------------------ --------------------------------------pod test-db-client-connectionstress-mssql-yfcvfw-------------------------------------- `kubectl logs test-db-client-connectionstress-mssql-yfcvfw --namespace ns-dqbkz --tail 500` at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) 11:08:25.980 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:08:27.494 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) 11:08:27.996 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more 11:08:29.510 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:08:30.012 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:08:31.526 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true 11:08:32.028 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more 11:08:33.542 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) 11:08:34.044 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:08:35.558 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: 11:08:36.060 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more 11:08:37.574 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:08:38.076 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:08:39.590 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true 11:08:40.092 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:08:41.606 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) 11:08:42.112 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:08:43.622 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true 11:08:44.124 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:08:45.638 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: 11:08:46.140 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:08:47.654 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true 11:08:48.156 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:09:15.909 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true 11:09:15.910 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more 11:09:15.911 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:09:17.446 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:09:17.949 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) 11:09:19.462 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:09:19.965 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) 11:09:21.478 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:09:21.980 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) 11:09:23.494 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:09:23.996 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: 11:09:25.510 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:09:26.012 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) 11:09:27.526 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:09:28.028 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) 11:09:29.543 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more 11:09:30.045 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) 11:09:31.558 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:09:32.060 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true ------------------------------------------------------------------------------------------------------------------ check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Running Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:02 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:01 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:00 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file Sqlcmd: Error: Microsoft ODBC Driver 18 for SQL Server : Login timeout expired. Sqlcmd: Error: Microsoft ODBC Driver 18 for SQL Server : TCP Provider: Error code 0x2749. Sqlcmd: Error: Microsoft ODBC Driver 18 for SQL Server : A network-related or instance-specific error has occurred while establishing a connection to mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online.. command terminated with exit code 1 check mssql synchronization state... Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file SYNCHRONIZING check mssql synchronization state... SYNCHRONIZED SYNCHRONIZING Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:11:14.630 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true 11:11:15.132 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more 11:11:16.646 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:11:17.148 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. 11:11:18.662 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true java.io.IOException: Failed to connect to SQLServer: at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) 11:11:19.164 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more 11:11:20.678 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true Failed to connect to SQLServer database: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". Trying with database SQLServer. java.io.IOException: Failed to connect to SQLServer: 11:11:21.180 [main] DEBUG com.yashandb.conf.ConnectionUrl -- JDBC URL must start with "jdbc:yasdb:" but was: jdbc:sqlserver://mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local:1433;databaseName=master;encrypt=true;trustServerCertificate=true at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:63) at com.apecloud.dbtester.tester.SQLServerTester.connectionStress(SQLServerTester.java:111) at com.apecloud.dbtester.tester.TestExecutor.executeTest(TestExecutor.java:34) at OneClient.executeTest(OneClient.java:105) at OneClient.main(OneClient.java:37) Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local, port 1433 has failed. Error: "Connection refused (Connection refused). Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.". at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:242) at com.microsoft.sqlserver.jdbc.SQLServerException.convertConnectExceptionToSQLServerException(SQLServerException.java:308) at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2593) at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:721) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3768) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3385) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1971) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1263) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(Unknown Source) at com.apecloud.dbtester.tester.SQLServerTester.connect(SQLServerTester.java:61) ... 4 more `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge pods test-db-client-connectionstress-mssql-yfcvfw --namespace ns-dqbkz ` pod/test-db-client-connectionstress-mssql-yfcvfw patched (no change) Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "test-db-client-connectionstress-mssql-yfcvfw" force deleted check failover pod name get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... failover pod name:mssql-yfcvfw-mssql-2 failover connectionstress Failure cluster update monitor KUBEBLOCKS NAMESPACE:kb-zkhtd get kubeblocks namespace done check addon prometheus enabled check addon prometheus enabled done check addon grafana enabled check addon grafana enabled done check addon apecloud-otel-collector enabled check addon apecloud-otel-collector enabled done kubectl patch cluster mssql-yfcvfw --namespace ns-dqbkz --type json -p '[***"op": "replace", "path": "/spec/componentSpecs/0/disableExporter", "value": false***]' cluster.apps.kubeblocks.io/mssql-yfcvfw patched check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Failed cluster_status:Abnormal cluster_status:Abnormal check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:02 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:01 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:00 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-2;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED create curl-jq pod mssql-yfcvfw-kb-curl `kubectl run --namespace kb-zkhtd mssql-yfcvfw-kb-curl --restart='Never' --image docker.io/apecloud/curl-jq:0.1.0 --command -- sleep infinity ` pod/mssql-yfcvfw-kb-curl created check pod status pod_status:NAME READY STATUS RESTARTS AGE mssql-yfcvfw-kb-curl 0/1 ContainerCreating 0 1s check pod mssql-yfcvfw-kb-curl status done `kubectl wait --for=condition=Ready pod mssql-yfcvfw-kb-curl --namespace kb-zkhtd` pod/mssql-yfcvfw-kb-curl condition met pod_status:NAME READY STATUS RESTARTS AGE mssql-yfcvfw-kb-curl 1/1 Running 0 6s kubectl exec -it mssql-yfcvfw-kb-curl --namespace kb-zkhtd -- curl -G "http://kb-addon-prometheus-server:80/api/v1/query" -d "query=sum(k8s_pod_cpu_limit)" `kubectl exec -it mssql-yfcvfw-kb-curl --namespace kb-zkhtd -- curl -G "http://kb-addon-prometheus-server:80/api/v1/query" -d "query=sum(k8s_pod_cpu_limit)"` Unable to use a TTY - input is not a terminal or the right kind of file % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 107 100 107 0 0 35738 0 --:--:-- --:--:-- --:--:-- 53500 monitor_result:***"status":"success","data":***"resultType":"vector","result":[***"metric":***,"value":[1750331691.109,"1.5"]***]*** k8s_pod_cpu_limit: 1750331691.109 monitor cluster Success `kubectl delete pod mssql-yfcvfw-kb-curl --namespace kb-zkhtd --force` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. pod "mssql-yfcvfw-kb-curl" force deleted check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Running Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:02 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:01 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:00 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED test failover fullcpu check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mssql-yfcvfw" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-fullcpu-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-0 mode: all stressors: cpu: workers: 100 load: 100 duration: 2m `kubectl apply -f test-chaos-mesh-fullcpu-mssql-yfcvfw.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-fullcpu-mssql-yfcvfw created apply test-chaos-mesh-fullcpu-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-fullcpu-mssql-yfcvfw.yaml` fullcpu chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Running Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:02 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:01 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:00 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-fullcpu-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mssql-yfcvfw" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-fullcpu-mssql-yfcvfw" not found check failover pod name failover pod name:mssql-yfcvfw-mssql-0 failover fullcpu Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success test failover podfailure check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mssql-yfcvfw" not found Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: PodChaos metadata: name: test-chaos-mesh-podfailure-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-0 mode: all action: pod-failure duration: 2m `kubectl apply -f test-chaos-mesh-podfailure-mssql-yfcvfw.yaml` podchaos.chaos-mesh.org/test-chaos-mesh-podfailure-mssql-yfcvfw created apply test-chaos-mesh-podfailure-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-podfailure-mssql-yfcvfw.yaml` podfailure chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Abnormal Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Abnormal cluster_status:Abnormal cluster_status:Abnormal cluster_status:Abnormal cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:02 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:01 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:00 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-2;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... get mssql primary pod name... check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge PodChaos test-chaos-mesh-podfailure-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mssql-yfcvfw" force deleted Error from server (NotFound): podchaos.chaos-mesh.org "test-chaos-mesh-podfailure-mssql-yfcvfw" not found check failover pod name failover pod name:mssql-yfcvfw-mssql-0 failover podfailure Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success cluster stop check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster stop mssql-yfcvfw --auto-approve --force=true --namespace ns-dqbkz ` OpsRequest mssql-yfcvfw-stop-jlgrq created successfully, you can view the progress: kbcli cluster describe-ops mssql-yfcvfw-stop-jlgrq -n ns-dqbkz check ops status `kbcli cluster list-ops mssql-yfcvfw --status all --namespace ns-dqbkz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-yfcvfw-stop-jlgrq ns-dqbkz Stop mssql-yfcvfw mssql Running 0/3 Jun 19,2025 19:21 UTC+0800 check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Stopped Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Stopped check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME check pod status done check ops status `kbcli cluster list-ops mssql-yfcvfw --status all --namespace ns-dqbkz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-yfcvfw-stop-jlgrq ns-dqbkz Stop mssql-yfcvfw mssql Succeed 3/3 Jun 19,2025 19:21 UTC+0800 check ops status done ops_status:mssql-yfcvfw-stop-jlgrq ns-dqbkz Stop mssql-yfcvfw mssql Succeed 3/3 Jun 19,2025 19:21 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mssql-yfcvfw-stop-jlgrq --namespace ns-dqbkz ` opsrequest.apps.kubeblocks.io/mssql-yfcvfw-stop-jlgrq patched `kbcli cluster delete-ops --name mssql-yfcvfw-stop-jlgrq --force --auto-approve --namespace ns-dqbkz ` OpsRequest mssql-yfcvfw-stop-jlgrq deleted cluster start check cluster status before ops check cluster status done cluster_status:Stopped `kbcli cluster start mssql-yfcvfw --force=true --namespace ns-dqbkz ` OpsRequest mssql-yfcvfw-start-klctv created successfully, you can view the progress: kbcli cluster describe-ops mssql-yfcvfw-start-klctv -n ns-dqbkz check ops status `kbcli cluster list-ops mssql-yfcvfw --status all --namespace ns-dqbkz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-yfcvfw-start-klctv ns-dqbkz Start mssql-yfcvfw mssql Running 0/3 Jun 19,2025 19:22 UTC+0800 check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:22 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-2;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED check ops status `kbcli cluster list-ops mssql-yfcvfw --status all --namespace ns-dqbkz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-yfcvfw-start-klctv ns-dqbkz Start mssql-yfcvfw mssql Succeed 3/3 Jun 19,2025 19:22 UTC+0800 check ops status done ops_status:mssql-yfcvfw-start-klctv ns-dqbkz Start mssql-yfcvfw mssql Succeed 3/3 Jun 19,2025 19:22 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mssql-yfcvfw-start-klctv --namespace ns-dqbkz ` opsrequest.apps.kubeblocks.io/mssql-yfcvfw-start-klctv patched `kbcli cluster delete-ops --name mssql-yfcvfw-start-klctv --force --auto-approve --namespace ns-dqbkz ` OpsRequest mssql-yfcvfw-start-klctv deleted `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success test failover networklossover check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mssql-yfcvfw" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networklossover-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-2 mode: all action: loss loss: loss: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networklossover-mssql-yfcvfw.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networklossover-mssql-yfcvfw created apply test-chaos-mesh-networklossover-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-networklossover-mssql-yfcvfw.yaml` networklossover chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:22 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file NOT NOT check mssql synchronization state... NOT NOT Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networklossover-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mssql-yfcvfw" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networklossover-mssql-yfcvfw" not found check failover pod name failover pod name:mssql-yfcvfw-mssql-0 failover networklossover Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success test failover oom check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-yfcvfw" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-0 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-mssql-yfcvfw.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mssql-yfcvfw created apply test-chaos-mesh-oom-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-oom-mssql-yfcvfw.yaml` check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Abnormal cluster_status:Abnormal check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:22 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-yfcvfw" force deleted Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-yfcvfw" not found check failover pod name failover pod name:mssql-yfcvfw-mssql-0 checking failover... `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-yfcvfw" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-yfcvfw" not found `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-yfcvfw" not found Error from server (NotFound): stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: StressChaos metadata: name: test-chaos-mesh-oom-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-0 mode: all stressors: memory: workers: 1 size: "100GB" oomScoreAdj: -1000 duration: 2m `kubectl apply -f test-chaos-mesh-oom-mssql-yfcvfw.yaml` stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mssql-yfcvfw created apply test-chaos-mesh-oom-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-oom-mssql-yfcvfw.yaml` check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:22 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-2;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge StressChaos test-chaos-mesh-oom-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. stresschaos.chaos-mesh.org "test-chaos-mesh-oom-mssql-yfcvfw" force deleted stresschaos.chaos-mesh.org/test-chaos-mesh-oom-mssql-yfcvfw patched check failover pod name:mssql-yfcvfw-mssql-2 failover oom Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success test failover timeoffset check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mssql-yfcvfw" not found Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: TimeChaos metadata: name: test-chaos-mesh-timeoffset-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-2 mode: all timeOffset: '-10m' clockIds: - CLOCK_REALTIME duration: 2m `kubectl apply -f test-chaos-mesh-timeoffset-mssql-yfcvfw.yaml` timechaos.chaos-mesh.org/test-chaos-mesh-timeoffset-mssql-yfcvfw created apply test-chaos-mesh-timeoffset-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-timeoffset-mssql-yfcvfw.yaml` timeoffset chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Running Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:20Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:22 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-2;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge TimeChaos test-chaos-mesh-timeoffset-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mssql-yfcvfw" force deleted Error from server (NotFound): timechaos.chaos-mesh.org "test-chaos-mesh-timeoffset-mssql-yfcvfw" not found check failover pod name failover pod name:mssql-yfcvfw-mssql-2 failover timeoffset Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success `kubectl get pvc -l app.kubernetes.io/instance=mssql-yfcvfw,apps.kubeblocks.io/component-name=mssql,apps.kubeblocks.io/vct-name=data --namespace ns-dqbkz ` cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in mssql-yfcvfw namespace. `kbcli cluster volume-expand mssql-yfcvfw --auto-approve --force=true --components mssql --volume-claim-templates data --storage 24Gi --namespace ns-dqbkz ` OpsRequest mssql-yfcvfw-volumeexpansion-dr4zs created successfully, you can view the progress: kbcli cluster describe-ops mssql-yfcvfw-volumeexpansion-dr4zs -n ns-dqbkz check ops status `kbcli cluster list-ops mssql-yfcvfw --status all --namespace ns-dqbkz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-yfcvfw-volumeexpansion-dr4zs ns-dqbkz VolumeExpansion mssql-yfcvfw mssql Running 0/3 Jun 19,2025 19:33 UTC+0800 check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:24Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:24Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:24Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:22 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-2;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED No resources found in mssql-yfcvfw namespace. check ops status `kbcli cluster list-ops mssql-yfcvfw --status all --namespace ns-dqbkz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-yfcvfw-volumeexpansion-dr4zs ns-dqbkz VolumeExpansion mssql-yfcvfw mssql Succeed 3/3 Jun 19,2025 19:33 UTC+0800 check ops status done ops_status:mssql-yfcvfw-volumeexpansion-dr4zs ns-dqbkz VolumeExpansion mssql-yfcvfw mssql Succeed 3/3 Jun 19,2025 19:33 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mssql-yfcvfw-volumeexpansion-dr4zs --namespace ns-dqbkz ` opsrequest.apps.kubeblocks.io/mssql-yfcvfw-volumeexpansion-dr4zs patched `kbcli cluster delete-ops --name mssql-yfcvfw-volumeexpansion-dr4zs --force --auto-approve --namespace ns-dqbkz ` OpsRequest mssql-yfcvfw-volumeexpansion-dr4zs deleted `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success test failover drainnode check node drain check node drain success kubectl get pod mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -o jsonpath='***.spec.nodeName***' get node name:ip-172-31-1-26.us-west-2.compute.internal success check if multiple pods are on the same node kubectl get pod mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -o jsonpath='***.spec.nodeName***' get node name:ip-172-31-3-9.us-west-2.compute.internal success kubectl get pod mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -o jsonpath='***.spec.nodeName***' get node name:ip-172-31-10-24.us-west-2.compute.internal success kubectl drain ip-172-31-1-26.us-west-2.compute.internal --delete-emptydir-data --ignore-daemonsets --force --grace-period 0 --timeout 60s node/ip-172-31-1-26.us-west-2.compute.internal cordoned Warning: ignoring DaemonSet-managed Pods: chaos-mesh/chaos-daemon-thqmj, kb-zkhtd/kb-addon-apecloud-otel-collector-77mf6, kube-system/aws-node-j9jgg, kube-system/ebs-csi-node-4x45b, kube-system/kube-proxy-lqnkt evicting pod ns-dqbkz/mssql-yfcvfw-mssql-2 pod/mssql-yfcvfw-mssql-2 evicted node/ip-172-31-1-26.us-west-2.compute.internal drained kubectl uncordon ip-172-31-1-26.us-west-2.compute.internal node/ip-172-31-1-26.us-west-2.compute.internal uncordoned check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:24Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:24Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:24Gi ip-172-31-1-27.us-west-2.compute.internal/172.31.1.27 Jun 19,2025 19:34 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED check failover pod name failover pod name:mssql-yfcvfw-mssql-0 failover drainnode Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success test failover networkduplicate check node drain check node drain success `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mssql-yfcvfw" not found Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mssql-yfcvfw" not found apiVersion: chaos-mesh.org/v1alpha1 kind: NetworkChaos metadata: name: test-chaos-mesh-networkduplicate-mssql-yfcvfw namespace: ns-dqbkz spec: selector: namespaces: - ns-dqbkz labelSelectors: apps.kubeblocks.io/pod-name: mssql-yfcvfw-mssql-0 mode: all action: duplicate duplicate: duplicate: '100' correlation: '100' direction: to duration: 2m `kubectl apply -f test-chaos-mesh-networkduplicate-mssql-yfcvfw.yaml` networkchaos.chaos-mesh.org/test-chaos-mesh-networkduplicate-mssql-yfcvfw created apply test-chaos-mesh-networkduplicate-mssql-yfcvfw.yaml Success `rm -rf test-chaos-mesh-networkduplicate-mssql-yfcvfw.yaml` networkduplicate chaos test waiting 120 seconds check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Running Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 500m / 500m 2Gi / 2Gi data:24Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:24Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:22 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 500m / 500m 2Gi / 2Gi data:24Gi ip-172-31-1-27.us-west-2.compute.internal/172.31.1.27 Jun 19,2025 19:34 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-0;secondary: mssql-yfcvfw-mssql-1 mssql-yfcvfw-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge NetworkChaos test-chaos-mesh-networkduplicate-mssql-yfcvfw --namespace ns-dqbkz ` Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mssql-yfcvfw" force deleted Error from server (NotFound): networkchaos.chaos-mesh.org "test-chaos-mesh-networkduplicate-mssql-yfcvfw" not found check failover pod name failover pod name:mssql-yfcvfw-mssql-0 failover networkduplicate Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-1 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success cluster vscale check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale mssql-yfcvfw --auto-approve --force=true --components mssql --cpu 600m --memory 2.1Gi --namespace ns-dqbkz ` OpsRequest mssql-yfcvfw-verticalscaling-bkjvs created successfully, you can view the progress: kbcli cluster describe-ops mssql-yfcvfw-verticalscaling-bkjvs -n ns-dqbkz check ops status `kbcli cluster list-ops mssql-yfcvfw --status all --namespace ns-dqbkz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-yfcvfw-verticalscaling-bkjvs ns-dqbkz VerticalScaling mssql-yfcvfw mssql Running 0/3 Jun 19,2025 19:39 UTC+0800 check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Updating Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:24Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:40 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:24Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:40 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:24Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:39 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-2;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED check ops status `kbcli cluster list-ops mssql-yfcvfw --status all --namespace ns-dqbkz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME mssql-yfcvfw-verticalscaling-bkjvs ns-dqbkz VerticalScaling mssql-yfcvfw mssql Succeed 2/3 Jun 19,2025 19:39 UTC+0800 check ops status done ops_status:mssql-yfcvfw-verticalscaling-bkjvs ns-dqbkz VerticalScaling mssql-yfcvfw mssql Succeed 2/3 Jun 19,2025 19:39 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests mssql-yfcvfw-verticalscaling-bkjvs --namespace ns-dqbkz ` opsrequest.apps.kubeblocks.io/mssql-yfcvfw-verticalscaling-bkjvs patched `kbcli cluster delete-ops --name mssql-yfcvfw-verticalscaling-bkjvs --force --auto-approve --namespace ns-dqbkz ` OpsRequest mssql-yfcvfw-verticalscaling-bkjvs deleted `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` check db_client batch data Success `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check readonly db_client batch data count `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Kreadonly -Q \"select count(*) from executions_loop..executions_loop_table;\" " | kubectl exec -it mssql-yfcvfw-mssql-0 --namespace ns-dqbkz -- bash` check readonly db_client batch data Success cluster update terminationPolicy WipeOut `kbcli cluster update mssql-yfcvfw --termination-policy=WipeOut --namespace ns-dqbkz ` cluster.apps.kubeblocks.io/mssql-yfcvfw updated (no change) check cluster status `kbcli cluster list mssql-yfcvfw --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw ns-dqbkz mssql WipeOut Running Jun 19,2025 18:13 UTC+0800 app.kubernetes.io/instance=mssql-yfcvfw,clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-mssql-0 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:24Gi ip-172-31-3-9.us-west-2.compute.internal/172.31.3.9 Jun 19,2025 19:40 UTC+0800 mssql-yfcvfw-mssql-1 ns-dqbkz mssql-yfcvfw mssql Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:24Gi ip-172-31-10-24.us-west-2.compute.internal/172.31.10.24 Jun 19,2025 19:40 UTC+0800 mssql-yfcvfw-mssql-2 ns-dqbkz mssql-yfcvfw mssql Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:24Gi ip-172-31-1-26.us-west-2.compute.internal/172.31.1.26 Jun 19,2025 19:39 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-mssql-2;secondary: mssql-yfcvfw-mssql-0 mssql-yfcvfw-mssql-1 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw` set secret: mssql-yfcvfw-mssql-account-sa `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-mssql-2 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED cluster full backup `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.name***"` `kubectl get backuprepo backuprepo-kbcli-test -o jsonpath="***.spec.credential.namespace***"` `kubectl get secrets kb-backuprepo-zxh9n -n kb-zkhtd -o jsonpath="***.data.accessKeyId***"` `kubectl get secrets kb-backuprepo-zxh9n -n kb-zkhtd -o jsonpath="***.data.secretAccessKey***"` KUBEBLOCKS NAMESPACE:kb-zkhtd get kubeblocks namespace done `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-zkhtd -o jsonpath="***.items[0].data.root-user***"` `kubectl get secrets -l app.kubernetes.io/instance=kbcli-test-minio --namespace kb-zkhtd -o jsonpath="***.items[0].data.root-password***"` minio_user:kbclitest,minio_password:kbclitest,minio_endpoint:kbcli-test-minio.kb-zkhtd.svc.cluster.local:9000 list minio bucket kbcli-test `echo 'mc config host add minioserver http://kbcli-test-minio.kb-zkhtd.svc.cluster.local:9000 kbclitest kbclitest;mc ls minioserver' | kubectl exec -it kbcli-test-minio-d9bd755d8-fxn5w --namespace kb-zkhtd -- bash` Unable to use a TTY - input is not a terminal or the right kind of file list minio bucket done default backuprepo:backuprepo-kbcli-test exists `kbcli cluster backup mssql-yfcvfw --method full --namespace ns-dqbkz ` Backup backup-ns-dqbkz-mssql-yfcvfw-20250619194257 created successfully, you can view the progress: kbcli cluster list-backups --name=backup-ns-dqbkz-mssql-yfcvfw-20250619194257 -n ns-dqbkz check backup status `kbcli cluster list-backups mssql-yfcvfw --namespace ns-dqbkz ` NAME NAMESPACE SOURCE-CLUSTER METHOD STATUS TOTAL-SIZE DURATION CREATE-TIME COMPLETION-TIME EXPIRATION backup-ns-dqbkz-mssql-yfcvfw-20250619194257 ns-dqbkz mssql-yfcvfw full Running Jun 19,2025 19:42 UTC+0800 backup_status:mssql-yfcvfw-full-Running backup_status:mssql-yfcvfw-full-Running backup_status:mssql-yfcvfw-full-Running check backup status done backup_status:backup-ns-dqbkz-mssql-yfcvfw-20250619194257 ns-dqbkz mssql-yfcvfw full Completed 6997011 15s Jun 19,2025 19:42 UTC+0800 Jun 19,2025 19:43 UTC+0800 cluster restore backup Error from server (NotFound): opsrequests.apps.kubeblocks.io "mssql-yfcvfw-backup" not found `kbcli cluster describe-backup backup-ns-dqbkz-mssql-yfcvfw-20250619194257 --namespace ns-dqbkz ` Name: backup-ns-dqbkz-mssql-yfcvfw-20250619194257 Cluster: mssql-yfcvfw Namespace: ns-dqbkz Spec: Method: full Policy Name: mssql-yfcvfw-mssql-backup-policy Status: Phase: Completed Total Size: 6997011 ActionSet Name: mssql-full-backup Repository: backuprepo-kbcli-test Duration: 15s Start Time: Jun 19,2025 19:42 UTC+0800 Completion Time: Jun 19,2025 19:43 UTC+0800 Path: /ns-dqbkz/mssql-yfcvfw-527fd339-2f94-4750-8eb8-25bb89e4f787/mssql/backup-ns-dqbkz-mssql-yfcvfw-20250619194257 Time Range End: Jun 19,2025 19:43 UTC+0800 Warning Events: `kbcli cluster restore mssql-yfcvfw-backup --backup backup-ns-dqbkz-mssql-yfcvfw-20250619194257 --namespace ns-dqbkz ` Cluster mssql-yfcvfw-backup created check cluster status `kbcli cluster list mssql-yfcvfw-backup --show-labels --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS mssql-yfcvfw-backup ns-dqbkz mssql WipeOut Jun 19,2025 19:43 UTC+0800 clusterdefinition.kubeblocks.io/name=mssql,clusterversion.kubeblocks.io/name= cluster_status: cluster_status: cluster_status: cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances mssql-yfcvfw-backup --namespace ns-dqbkz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME mssql-yfcvfw-backup-mssql-0 ns-dqbkz mssql-yfcvfw-backup mssql Running primary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:24Gi ip-172-31-14-103.us-west-2.compute.internal/172.31.14.103 Jun 19,2025 19:43 UTC+0800 mssql-yfcvfw-backup-mssql-1 ns-dqbkz mssql-yfcvfw-backup mssql Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:24Gi ip-172-31-1-27.us-west-2.compute.internal/172.31.1.27 Jun 19,2025 19:43 UTC+0800 mssql-yfcvfw-backup-mssql-2 ns-dqbkz mssql-yfcvfw-backup mssql Running secondary us-west-2a 600m / 600m 2254857830400m / 2254857830400m data:24Gi ip-172-31-3-103.us-west-2.compute.internal/172.31.3.103 Jun 19,2025 19:43 UTC+0800 check pod status done check cluster role check cluster role done primary: mssql-yfcvfw-backup-mssql-0;secondary: mssql-yfcvfw-backup-mssql-1 mssql-yfcvfw-backup-mssql-2 `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw-backup` set secret: mssql-yfcvfw-backup-mssql-account-sa `kubectl get secrets mssql-yfcvfw-backup-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-backup-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-backup-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. check cluster connect `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-backup-mssql-mssql-ro.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"select synchronization_state_desc from sys.dm_hadr_database_replica_states;\" " | kubectl exec -it mssql-yfcvfw-backup-mssql-0 --namespace ns-dqbkz -- bash` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file check mssql synchronization state success SYNCHRONIZED SYNCHRONIZED `kbcli cluster describe-backup backup-ns-dqbkz-mssql-yfcvfw-20250619194257 --namespace ns-dqbkz ` Name: backup-ns-dqbkz-mssql-yfcvfw-20250619194257 Cluster: mssql-yfcvfw Namespace: ns-dqbkz Spec: Method: full Policy Name: mssql-yfcvfw-mssql-backup-policy Status: Phase: Completed Total Size: 6997011 ActionSet Name: mssql-full-backup Repository: backuprepo-kbcli-test Duration: 15s Start Time: Jun 19,2025 19:42 UTC+0800 Completion Time: Jun 19,2025 19:43 UTC+0800 Path: /ns-dqbkz/mssql-yfcvfw-527fd339-2f94-4750-8eb8-25bb89e4f787/mssql/backup-ns-dqbkz-mssql-yfcvfw-20250619194257 Time Range End: Jun 19,2025 19:43 UTC+0800 Warning Events: cluster connect `kubectl get secrets -l app.kubernetes.io/instance=mssql-yfcvfw-backup` set secret: mssql-yfcvfw-backup-mssql-account-sa `kubectl get secrets mssql-yfcvfw-backup-mssql-account-sa -o jsonpath="***.data.username***"` `kubectl get secrets mssql-yfcvfw-backup-mssql-account-sa -o jsonpath="***.data.password***"` `kubectl get secrets mssql-yfcvfw-backup-mssql-account-sa -o jsonpath="***.data.port***"` DB_USERNAME:sa;DB_PASSWORD:98Tmq4hsgz;DB_PORT:1433;DB_DATABASE: No resources found in ns-dqbkz namespace. `echo "/opt/mssql-tools18/bin/sqlcmd -S mssql-yfcvfw-backup-mssql-mssql-rw.ns-dqbkz.svc.cluster.local,1433 -U sa -P '98Tmq4hsgz' -C -Q \"SELECT ag.name AS AvailabilityGroupName,ar.replica_server_name AS ReplicaName,ars.role_desc AS Role,ars.synchronization_health_desc AS SyncHealth FROM sys.availability_groups ag JOIN sys.availability_replicas ar ON ag.group_id = ar.group_id JOIN sys.dm_hadr_availability_replica_states ars ON ar.replica_id = ars.replica_id;\" " | kubectl exec -it mssql-yfcvfw-backup-mssql-0 --namespace ns-dqbkz -- bash ` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) Unable to use a TTY - input is not a terminal or the right kind of file AvailabilityGroupName ReplicaName Role SyncHealth -------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------ ------------------------------------------------------------ ag1 mssql-yfcvfw-backup-mssql-0 PRIMARY HEALTHY ag1 mssql-yfcvfw-backup-mssql-1 SECONDARY HEALTHY ag1 mssql-yfcvfw-backup-mssql-2 SECONDARY HEALTHY (3 rows affected) connect cluster Success delete cluster mssql-yfcvfw-backup `kbcli cluster delete mssql-yfcvfw-backup --auto-approve --namespace ns-dqbkz ` Cluster mssql-yfcvfw-backup deleted Error from server (NotFound): secrets "mssql-yfcvfw-backup-certificates" not found Error from server (NotFound): secrets "mssql-yfcvfw-backup-certificates" not found Error from server (NotFound): secrets "mssql-yfcvfw-backup-certificates" not found No resources found in ns-dqbkz namespace. delete cluster pod done No resources found in ns-dqbkz namespace. check cluster resource non-exist OK: pvc No resources found in ns-dqbkz namespace. delete cluster done No resources found in ns-dqbkz namespace. No resources found in ns-dqbkz namespace. No resources found in ns-dqbkz namespace. cluster delete backup `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge backups backup-ns-dqbkz-mssql-yfcvfw-20250619194257 --namespace ns-dqbkz ` backup.dataprotection.kubeblocks.io/backup-ns-dqbkz-mssql-yfcvfw-20250619194257 patched `kbcli cluster delete-backup mssql-yfcvfw --name backup-ns-dqbkz-mssql-yfcvfw-20250619194257 --force --auto-approve --namespace ns-dqbkz ` Backup backup-ns-dqbkz-mssql-yfcvfw-20250619194257 deleted No opsrequests found in ns-dqbkz namespace. cluster list-logs `kbcli cluster list-logs mssql-yfcvfw --namespace ns-dqbkz ` No log files found. You can enable the log feature with the kbcli command below. kbcli cluster update mssql-yfcvfw --enable-all-logs=true --namespace ns-dqbkz Error from server (NotFound): pods "mssql-yfcvfw-mssql-2" not found cluster logs `kbcli cluster logs mssql-yfcvfw --tail 30 --namespace ns-dqbkz ` Defaulted container "mssql" out of: mssql, exporter, lorry, config-manager, init-lorry (init), install-config-manager-tool (init) AGID [BE9BE8C9-6E17-1132-BFBA-8B7D2C28AFDB] : [15AF26B8-18DE-4DF6-9C7B-DB49C36072B1] local replica received build replica response from [566DC216-FA4F-4AD0-9115-F6D406B81D43], Response code [0x80071c27]. Message handle status [1].2025-06-19 11:41:54.25 spid86s Always On: DebugTraceVarArgs AR '[HADR] [Primary] operation on replicas [15AF26B8-18DE-4DF6-9C7B-DB49C36072B1]->[566DC216-FA4F-4AD0-9115-F6D406B81D43], database [db1], remote endpoint [tcp://mssql-yfcvfw-mssql-1.mssql-yfcvfw-mssql-headless.ns-dqbkz.svc.cluster.local:5022], source operation [CF1A49AC-0286-4639-B2CF-28D44A513079]: Transitioning from [CHECK_IF_SEEDING_NEEDED] to [CATCHUP].' [HADR TRANSPORT] AR[15AF26B8-18DE-4DF6-9C7B-DB49C36072B1]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E] Dropping Timeout Task with Invalid Generation[HADR TRANSPORT] AR[15AF26B8-18DE-4DF6-9C7B-DB49C36072B1]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Dropping Timeout Task with Invalid Generation[HADR TRANSPORT] AR[15AF26B8-18DE-4DF6-9C7B-DB49C36072B1]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E] Dropping Reconnect Task with Invalid Generation2025-06-19 11:41:57.14 spid80s Always On: *** XClusterPaxos [BE9BE8C9-6E17-1132-BFBA-8B7D2C28AFDB] GetResult. GetSequenceNumber =1f00000007 Failed hr = 0 2025-06-19 11:41:57.15 spid80s Always On Availability Groups connection with secondary database established for primary database 'db1' on the availability replica 'mssql-yfcvfw-mssql-1' with Replica ID: ***566dc216-fa4f-4ad0-9115-f6d406b81d43***. This is an informational message only. No user action is required. 2025-06-19 11:41:57.17 spid63s Always On: *** XClusterPaxos [BE9BE8C9-6E17-1132-BFBA-8B7D2C28AFDB] GetResult. GetSequenceNumber =1f00000008 Failed hr = 0 2025-06-19 11:41:57.17 spid63s Always On Availability Groups connection with secondary database established for primary database 'executions_loop' on the availability replica 'mssql-yfcvfw-mssql-1' with Replica ID: ***566dc216-fa4f-4ad0-9115-f6d406b81d43***. This is an informational message only. No user action is required. DbMgrPartner::InitLogBase [5]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Enabling new compression support.HadrLogCapture::LogPoolStartScan: DbId[5]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] start with log block id 0x0000002a0000011d.HadrReplicaOnlineManager::HandleLogScanStateChange[5]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Handling change state from 1 to 5LOG_SEND_TRANSITION: DbId [5] -> ReplicaId [566DC216-FA4F-4AD0-9115-F6D406B81D43] switched to LogWriter at log block id: 0x0000002a000001202025-06-19 11:41:57.26 spid90s Always On Availability Groups connection with secondary database established for primary database 'db1' on the availability replica 'mssql-yfcvfw-mssql-1' with Replica ID: ***566dc216-fa4f-4ad0-9115-f6d406b81d43***. This is an informational message only. No user action is required. HadrReplicaOnlineManager::HandleLogScanStateChange[5]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Update cluster registry key 'ReplicaStatus' to [1]HadrReplicaOnlineManager::HandleLogScanStateChange[5]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Handling change state from 5 to 0DbMgrPartner::InitLogBase [6]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Enabling new compression support.HadrLogCapture::LogPoolStartScan: DbId[6]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] start with log block id 0x0000002d00000032.LOG_SEND_TRANSITION: DbId [6] -> ReplicaId [566DC216-FA4F-4AD0-9115-F6D406B81D43] switched to LogWriter at log block id: 0x0000002d00000035HadrReplicaOnlineManager::HandleLogScanStateChange[5]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Update cluster registry key 'ReplicaStatus' to [0]2025-06-19 11:41:57.28 spid40s Always On Availability Groups connection with secondary database established for primary database 'executions_loop' on the availability replica 'mssql-yfcvfw-mssql-1' with Replica ID: ***566dc216-fa4f-4ad0-9115-f6d406b81d43***. This is an informational message only. No user action is required. HadrReplicaOnlineManager::HandleLogScanStateChange[6]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Handling change state from 1 to 5HadrReplicaOnlineManager::HandleLogScanStateChange[6]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Update cluster registry key 'ReplicaStatus' to [1]HadrReplicaOnlineManager::HandleLogScanStateChange[6]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Handling change state from 5 to 0HadrReplicaOnlineManager::HandleLogScanStateChange[6]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Update cluster registry key 'ReplicaStatus' to [0]DbMgrPartner::InitLogBase [5]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Enabling new compression support.HadrLogCapture::LogPoolStartScan: DbId[5]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] start with log block id 0x0000002a0000011d.HadrReplicaOnlineManager::HandleLogScanStateChange[5]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Handling change state from 1 to 5HadrReplicaOnlineManager::HandleLogScanStateChange[5]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Update cluster registry key 'ReplicaStatus' to [1]2025-06-19 11:41:57.35 spid97s Always On: DebugTraceVarArgs AR '[HADR] [Primary] operation on replicas [15AF26B8-18DE-4DF6-9C7B-DB49C36072B1]->[566DC216-FA4F-4AD0-9115-F6D406B81D43], database [db1], remote endpoint [tcp://mssql-yfcvfw-mssql-1.mssql-yfcvfw-mssql-headless.ns-dqbkz.svc.cluster.local:5022], source operation [CF1A49AC-0286-4639-B2CF-28D44A513079]: Transitioning from [CATCHUP] to [COMPLETED].' DbMgrPartner::InitLogBase [6]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Enabling new compression support.HadrLogCapture::LogPoolStartScan: DbId[6]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] start with log block id 0x0000002d00000032.HadrReplicaOnlineManager::HandleLogScanStateChange[6]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Handling change state from 1 to 5HadrReplicaOnlineManager::HandleLogScanStateChange[6]->[566DC216-FA4F-4AD0-9115-F6D406B81D43] Update cluster registry key 'ReplicaStatus' to [1]2025-06-19 11:41:57.39 spid64s Always On: *** XClusterPaxos [BE9BE8C9-6E17-1132-BFBA-8B7D2C28AFDB] GetResult. GetSequenceNumber =1f00000009 Failed hr = 0 2025-06-19 11:41:57.39 spid64s [DbMgrPartnerCommitPolicy::SetReplicaInfoInAg] DbMgrPartnerCommitPolicy::SetSyncAndRecoveryPoint: 566DC216-FA4F-4AD0-9115-F6D406B81D43:5:4 2025-06-19 11:41:57.40 spid64s DbMgrPartnerCommitPolicy::SetSyncState: 566DC216-FA4F-4AD0-9115-F6D406B81D43:5:4 2025-06-19 11:41:57.40 spid87s Always On: DebugTraceVarArgs AR '[HADR] [Primary] operation on replicas [15AF26B8-18DE-4DF6-9C7B-DB49C36072B1]->[566DC216-FA4F-4AD0-9115-F6D406B81D43], database [executions_loop], remote endpoint [tcp://mssql-yfcvfw-mssql-1.mssql-yfcvfw-mssql-headless.ns-dqbkz.svc.cluster.local:5022], source operation [E2175508-9880-4C20-B4C8-D7FB4D4C3AB7]: Transitioning from [CATCHUP] to [COMPLETED].' 2025-06-19 11:41:57.41 spid67s Always On: *** XClusterPaxos [BE9BE8C9-6E17-1132-BFBA-8B7D2C28AFDB] GetResult. GetSequenceNumber =1f0000000a Failed hr = 0 2025-06-19 11:41:57.41 spid67s [DbMgrPartnerCommitPolicy::SetReplicaInfoInAg] DbMgrPartnerCommitPolicy::SetSyncAndRecoveryPoint: 566DC216-FA4F-4AD0-9115-F6D406B81D43:6:4 2025-06-19 11:41:57.42 spid67s DbMgrPartnerCommitPolicy::SetSyncState: 566DC216-FA4F-4AD0-9115-F6D406B81D43:6:4 2025-06-19 11:42:09.07 spid64s Always On: *** XClusterPaxos [BE9BE8C9-6E17-1132-BFBA-8B7D2C28AFDB] GetResult. GetSequenceNumber =1f0000000b Failed hr = 0 2025-06-19 11:42:09.08 spid64s Always On Availability Groups connection with secondary database established for primary database 'executions_loop' on the availability replica 'mssql-yfcvfw-mssql-0' with Replica ID: ***dc0bd0ea-4101-42c4-91a6-e9c76bf04d1e***. This is an informational message only. No user action is required. AGID [BE9BE8C9-6E17-1132-BFBA-8B7D2C28AFDB] : [15AF26B8-18DE-4DF6-9C7B-DB49C36072B1] local replica received build replica response from [DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E], Response code [0x80071c27]. Message handle status [1].2025-06-19 11:42:09.13 spid89s Always On: DebugTraceVarArgs AR '[HADR] [Primary] operation on replicas [15AF26B8-18DE-4DF6-9C7B-DB49C36072B1]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E], database [executions_loop], remote endpoint [tcp://mssql-yfcvfw-mssql-0.mssql-yfcvfw-mssql-headless.ns-dqbkz.svc.cluster.local:5022], source operation [6A1DBA02-19BE-4FF9-B162-938EB0DF69B1]: Transitioning from [CHECK_IF_SEEDING_NEEDED] to [CATCHUP].' 2025-06-19 11:42:09.16 spid7s Always On: *** XClusterPaxos [BE9BE8C9-6E17-1132-BFBA-8B7D2C28AFDB] GetResult. GetSequenceNumber =1f0000000c Failed hr = 0 AGID [BE9BE8C9-6E17-1132-BFBA-8B7D2C28AFDB] : [15AF26B8-18DE-4DF6-9C7B-DB49C36072B1] local replica received build replica response from [DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E], Response code [0x80071c27]. Message handle status [1].2025-06-19 11:42:09.16 spid88s Always On: DebugTraceVarArgs AR '[HADR] [Primary] operation on replicas [15AF26B8-18DE-4DF6-9C7B-DB49C36072B1]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E], database [db1], remote endpoint [tcp://mssql-yfcvfw-mssql-0.mssql-yfcvfw-mssql-headless.ns-dqbkz.svc.cluster.local:5022], source operation [A728CDB6-AC52-467B-A59D-A78D3F3543E1]: Transitioning from [CHECK_IF_SEEDING_NEEDED] to [CATCHUP].' 2025-06-19 11:42:09.16 spid7s Always On Availability Groups connection with secondary database established for primary database 'db1' on the availability replica 'mssql-yfcvfw-mssql-0' with Replica ID: ***dc0bd0ea-4101-42c4-91a6-e9c76bf04d1e***. This is an informational message only. No user action is required. DbMgrPartner::InitLogBase [6]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E] Enabling new compression support.DbMgrPartner::InitLogBase [5]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E] Enabling new compression support.HadrLogCapture::LogPoolStartScan: DbId[5]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E] start with log block id 0x0000002a0000011d.HadrReplicaOnlineManager::HandleLogScanStateChange[5]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E] Handling change state from 1 to 5LOG_SEND_TRANSITION: DbId [5] -> ReplicaId [DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E] switched to LogWriter at log block id: 0x0000002a00000120HadrLogCapture::LogPoolStartScan: DbId[6]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E] start with log block id 0x0000002d00000032.LOG_SEND_TRANSITION: DbId [6] -> ReplicaId [DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E] switched to LogWriter at log block id: 0x0000002d00000035HadrReplicaOnlineManager::HandleLogScanStateChange[5]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E] Update cluster registry key 'ReplicaStatus' to [1]2025-06-19 11:42:09.48 spid87s Always On: DebugTraceVarArgs AR '[HADR] [Primary] operation on replicas [15AF26B8-18DE-4DF6-9C7B-DB49C36072B1]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E], database [db1], remote endpoint [tcp://mssql-yfcvfw-mssql-0.mssql-yfcvfw-mssql-headless.ns-dqbkz.svc.cluster.local:5022], source operation [A728CDB6-AC52-467B-A59D-A78D3F3543E1]: Transitioning from [CATCHUP] to [COMPLETED].' 2025-06-19 11:42:09.49 spid86s Always On: DebugTraceVarArgs AR '[HADR] [Primary] operation on replicas [15AF26B8-18DE-4DF6-9C7B-DB49C36072B1]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E], database [executions_loop], remote endpoint [tcp://mssql-yfcvfw-mssql-0.mssql-yfcvfw-mssql-headless.ns-dqbkz.svc.cluster.local:5022], source operation [6A1DBA02-19BE-4FF9-B162-938EB0DF69B1]: Transitioning from [CATCHUP] to [COMPLETED].' 2025-06-19 11:42:09.49 spid64s Always On: *** XClusterPaxos [BE9BE8C9-6E17-1132-BFBA-8B7D2C28AFDB] GetResult. GetSequenceNumber =1f0000000d Failed hr = 0 HadrReplicaOnlineManager::HandleLogScanStateChange[6]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E] Handling change state from 1 to 5HadrReplicaOnlineManager::HandleLogScanStateChange[6]->[DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E] Update cluster registry key 'ReplicaStatus' to [1]2025-06-19 11:42:09.50 spid64s [DbMgrPartnerCommitPolicy::SetReplicaInfoInAg] DbMgrPartnerCommitPolicy::SetSyncAndRecoveryPoint: DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E:5:4 2025-06-19 11:42:09.58 spid64s DbMgrPartnerCommitPolicy::SetSyncState: DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E:5:4 2025-06-19 11:42:09.59 spid88s Always On: *** XClusterPaxos [BE9BE8C9-6E17-1132-BFBA-8B7D2C28AFDB] GetResult. GetSequenceNumber =1f0000000e Failed hr = 0 2025-06-19 11:42:09.60 spid88s [DbMgrPartnerCommitPolicy::SetReplicaInfoInAg] DbMgrPartnerCommitPolicy::SetSyncAndRecoveryPoint: DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E:6:4 2025-06-19 11:42:09.60 spid88s DbMgrPartnerCommitPolicy::SetSyncState: DC0BD0EA-4101-42C4-91A6-E9C76BF04D1E:6:4 Force log send mode into LogCapture.Force log send mode into LogCapture.Backup certificate to PFX format with AES_256 encryption delete cluster mssql-yfcvfw `kbcli cluster delete mssql-yfcvfw --auto-approve --namespace ns-dqbkz ` Cluster mssql-yfcvfw deleted secret "mssql-yfcvfw-certificates" deleted pod_info:mssql-yfcvfw-mssql-0 0/4 Completed 0 5m35s mssql-yfcvfw-mssql-1 0/4 Completed 0 6m8s mssql-yfcvfw-mssql-2 0/4 Completed 0 6m48s No resources found in ns-dqbkz namespace. delete cluster pod done No resources found in ns-dqbkz namespace. check cluster resource non-exist OK: pvc No resources found in ns-dqbkz namespace. delete cluster done No resources found in ns-dqbkz namespace. No resources found in ns-dqbkz namespace. No resources found in ns-dqbkz namespace. Mssql Test Suite All Done! --------------------------------------Mssql (Topology = cluster Replicas 3) Test Result-------------------------------------- [PASSED]|[Create]|[Topology=cluster;ComponentVersion=mssql;ServiceVersion=2022.19.0;]|[Description=Create a cluster with the specified topology cluster and component version mssql and service version 2022.19.0] [PASSED]|[Connect]|[ComponentName=mssql]|[Description=Connect to the cluster] [PASSED]|[Failover]|[HA=DNS Random;Durations=2m;ComponentName=mssql]|[Description=Simulates conditions where pods experience random IP addresses being returned by the DNS service for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to the DNS service returning random IP addresses.] [FAILED]|[Failover]|[HA=Network Partition;Durations=2m;ComponentName=mssql]|[Description=Simulates network partition fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to partition network.] [PASSED]|[Failover]|[HA=Network Bandwidth Failover;Durations=2m;ComponentName=mssql]|[Description=Simulates network bandwidth fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to bandwidth network.] [PASSED]|[Failover]|[HA=Network Delay;Durations=2m;ComponentName=mssql]|[Description=Simulates network delay fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to delay network.] [PASSED]|[Failover]|[HA=DNS Error;Durations=2m;ComponentName=mssql]|[Description=Simulates conditions where pods experience DNS service errors for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to DNS service errors.] [PASSED]|[SwitchOver]|[ComponentName=mssql]|[Description=SwitchOver the cluster specify component mssql] [PASSED]|[Failover]|[HA=Network Corrupt Failover;Durations=2m;ComponentName=mssql]|[Description=Simulates network corrupt fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to corrupt network.] [FAILED]|[Failover]|[HA=Delete Pod;ComponentName=mssql]|[Description=Simulates conditions where pods terminating forced/graceful thereby testing deployment sanity (replica availability & uninterrupted service) and recovery workflow of the application.] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [FAILED]|[Failover]|[HA=Connection Stress;ComponentName=mssql]|[Description=Simulates conditions where pods experience connection stress either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Connection load.] [PASSED]|[Check Monitor]|[Enable=true]|[Description=Check Monitor Enable] [PASSED]|[Update]|[Monitor=true]|[Description=Update the cluster Monitor enable] [PASSED]|[Failover]|[HA=Full CPU;Durations=2m;ComponentName=mssql]|[Description=Simulates conditions where pods experience CPU full either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high CPU load.] [PASSED]|[Failover]|[HA=Pod Failure;Durations=2m;ComponentName=mssql]|[Description=Simulates conditions where pods experience failure for a period of time either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to failure.] [PASSED]|[Stop]|[-]|[Description=Stop the cluster] [PASSED]|[Start]|[-]|[Description=Start the cluster] [PASSED]|[Failover]|[HA=Network Loss Failover;Durations=2m;ComponentName=mssql]|[Description=Simulates network loss fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to loss network.] [PASSED]|[Failover]|[HA=OOM;Durations=2m;ComponentName=mssql]|[Description=Simulates conditions where pods experience OOM either due to expected/undesired processes thereby testing the application's resilience to potential slowness/unavailability of some replicas due to high Memory load.] [PASSED]|[Failover]|[HA=Time Offset;Durations=2m;ComponentName=mssql]|[Description=Simulates a time offset scenario thereby testing the application's resilience to potential slowness/unavailability of some replicas due to time offset.] [PASSED]|[VolumeExpansion]|[ComponentName=mssql]|[Description=VolumeExpansion the cluster specify component mssql] [PASSED]|[Failover]|[HA=Evicting Pod;ComponentName=mssql]|[Description=Simulates conditions where pods evicting either due to node drained thereby testing the application's resilience to unavailability of some replicas due to evicting.] [PASSED]|[Failover]|[HA=Network Duplicate;Durations=2m;ComponentName=mssql]|[Description=Simulates network duplicate fault thereby testing the application's resilience to potential slowness/unavailability of some replicas due to duplicate network.] [PASSED]|[VerticalScaling]|[ComponentName=mssql]|[Description=VerticalScaling the cluster specify component mssql] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Backup]|[BackupMethod=full]|[Description=The cluster full Backup] [PASSED]|[Restore]|[BackupMethod=full]|[Description=The cluster full Restore] [PASSED]|[Connect]|[ComponentName=mssql]|[Description=Connect to the cluster] [PASSED]|[Delete Restore Cluster]|[BackupMethod=full]|[Description=Delete the full restore cluster] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]