bash test/kbcli/test_kbcli_0.9.sh --type 11 --version 0.9.5 --generate-output true --chaos-mesh true --drain-node true --random-namespace true --region eastus --cloud-provider aks CURRENT_TEST_DIR:test/kbcli source commons files source engines files source kubeblocks files `kubectl get namespace | grep ns-oxzrz ` `kubectl create namespace ns-oxzrz` namespace/ns-oxzrz created create namespace ns-oxzrz done download kbcli `gh release list --repo apecloud/kbcli --limit 100 | (grep "0.9" || true)` `curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v0.9.5-beta.8` Your system is linux_amd64 Installing kbcli ... Downloading ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 32.1M 100 32.1M 0 0 136M 0 --:--:-- --:--:-- --:--:-- 136M kbcli installed successfully. Kubernetes: v1.32.6 KubeBlocks: 0.9.5 kbcli: 0.9.5-beta.8 WARNING: version difference between kbcli (0.9.5-beta.8) and kubeblocks (0.9.5) Make sure your docker service is running and begin your journey with kbcli: kbcli playground init For more information on how to get started, please visit: https://kubeblocks.io download kbcli v0.9.5-beta.8 done Kubernetes: v1.32.6 KubeBlocks: 0.9.5 kbcli: 0.9.5-beta.8 WARNING: version difference between kbcli (0.9.5-beta.8) and kubeblocks (0.9.5) Kubernetes Env: v1.32.6 POD_RESOURCES: aks kb-default-sc found aks default-vsc found found default storage class: default kubeblocks version is:0.9.5 skip upgrade kubeblocks Error: no repositories to show helm repo add chaos-mesh https://charts.chaos-mesh.org "chaos-mesh" has been added to your repositories add helm chart repo chaos-mesh success chaos mesh already installed check cluster definition set component name:frontend set component version No resources found no component version found unsupported component definition REPORT_COUNT 0:0 not found component version set replicas first:2 set replicas third:2 set minimum cmpv service version set minimum cmpv service version replicas:2 REPORT_COUNT:1 CLUSTER_TOPOLOGY: Not found topology in cluster definition greptimedb LIMIT_CPU:0.1 LIMIT_MEMORY:0.5 storage size: 1 No resources found in ns-oxzrz namespace. termination_policy:WipeOut create 2 replica WipeOut greptimedb cluster check cluster version check cluster definition apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: name: greptime-jzuuke namespace: ns-oxzrz spec: clusterDefinitionRef: greptimedb clusterVersionRef: greptimedb-0.3.2 terminationPolicy: WipeOut componentSpecs: - name: frontend componentDefRef: frontend replicas: 2 resources: requests: cpu: 100m memory: 0.5Gi limits: cpu: 100m memory: 0.5Gi - name: datanode componentDefRef: datanode replicas: 2 resources: requests: cpu: 100m memory: 0.5Gi limits: cpu: 100m memory: 0.5Gi volumeClaimTemplates: - name: datanode spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi - name: meta componentDefRef: meta replicas: 1 resources: requests: cpu: 100m memory: 0.5Gi limits: cpu: 100m memory: 0.5Gi - name: etcd componentDefRef: etcd replicas: 3 resources: requests: cpu: 100m memory: 0.5Gi limits: cpu: 100m memory: 0.5Gi volumeClaimTemplates: - name: etcd-storage spec: storageClassName: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi `kubectl apply -f test_create_greptime-jzuuke.yaml` cluster.apps.kubeblocks.io/greptime-jzuuke created apply test_create_greptime-jzuuke.yaml Success `rm -rf test_create_greptime-jzuuke.yaml` check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Sep 01,2025 11:18 UTC+0800 clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status: cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Creating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 100m / 100m 512Mi / 512Mi datanode:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 100m / 100m 512Mi / 512Mi datanode:1Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:21 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running follower 0 100m / 100m 512Mi / 512Mi etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running follower 0 100m / 100m 512Mi / 512Mi etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running leader 0 100m / 100m 512Mi / 512Mi etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:21 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 check pod status done check cluster connect `echo 'greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash` check cluster connect done `kubectl get secrets -l app.kubernetes.io/instance=greptime-jzuuke` `kubectl get secrets greptime-jzuuke-conn-credential -o jsonpath="***.data.username***"` `kubectl get secrets greptime-jzuuke-conn-credential -o jsonpath="***.data.password***"` `kubectl get secrets greptime-jzuuke-conn-credential -o jsonpath="***.data.port***"` DB_USERNAME:root;DB_PASSWORD:vmzsfrph;DB_PORT:4002;DB_DATABASE: check pod greptime-jzuuke-frontend-0 container_name frontend exist password vmzsfrph No container logs contain secret password. describe cluster `kbcli cluster describe greptime-jzuuke --namespace ns-oxzrz ` Name: greptime-jzuuke Created Time: Sep 01,2025 11:18 UTC+0800 NAMESPACE CLUSTER-DEFINITION VERSION STATUS TERMINATION-POLICY ns-oxzrz greptimedb greptimedb-0.3.2 Running WipeOut Endpoints: COMPONENT MODE INTERNAL EXTERNAL frontend ReadWrite greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4000 greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001 greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4002 greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4003 greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4242 datanode ReadWrite greptime-jzuuke-datanode.ns-oxzrz.svc.cluster.local:4000 greptime-jzuuke-datanode.ns-oxzrz.svc.cluster.local:4001 meta ReadWrite greptime-jzuuke-meta.ns-oxzrz.svc.cluster.local:3002 greptime-jzuuke-meta.ns-oxzrz.svc.cluster.local:4000 etcd ReadWrite greptime-jzuuke-etcd.ns-oxzrz.svc.cluster.local:2379 greptime-jzuuke-etcd.ns-oxzrz.svc.cluster.local:2380 Topology: COMPONENT INSTANCE ROLE STATUS AZ NODE CREATED-TIME datanode greptime-jzuuke-datanode-0 Running 0 aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 datanode greptime-jzuuke-datanode-1 Running 0 aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:21 UTC+0800 etcd greptime-jzuuke-etcd-0 follower Running 0 aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 etcd greptime-jzuuke-etcd-1 follower Running 0 aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 etcd greptime-jzuuke-etcd-2 leader Running 0 aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 frontend greptime-jzuuke-frontend-0 Running 0 aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 frontend greptime-jzuuke-frontend-1 Running 0 aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:21 UTC+0800 meta greptime-jzuuke-meta-0 Running 0 aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 Resources Allocation: COMPONENT DEDICATED CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE-SIZE STORAGE-CLASS frontend false 100m / 100m 512Mi / 512Mi datanode false 100m / 100m 512Mi / 512Mi datanode:1Gi default meta false 100m / 100m 512Mi / 512Mi etcd false 100m / 100m 512Mi / 512Mi etcd-storage:1Gi default Images: COMPONENT TYPE IMAGE frontend frontend docker.io/apecloud/greptimedb:0.3.2 datanode datanode docker.io/apecloud/greptimedb:0.3.2 meta meta docker.io/apecloud/greptimedb:0.3.2 etcd etcd docker.io/apecloud/etcd:v3.5.5 Show cluster events: kbcli cluster list-events -n ns-oxzrz greptime-jzuuke `kbcli cluster label greptime-jzuuke app.kubernetes.io/instance- --namespace ns-oxzrz ` label "app.kubernetes.io/instance" not found. `kbcli cluster label greptime-jzuuke app.kubernetes.io/instance=greptime-jzuuke --namespace ns-oxzrz ` `kbcli cluster label greptime-jzuuke --list --namespace ns-oxzrz ` NAME NAMESPACE LABELS greptime-jzuuke ns-oxzrz app.kubernetes.io/instance=greptime-jzuuke clusterdefinition.kubeblocks.io/name=greptimedb clusterversion.kubeblocks.io/name=greptimedb-0.3.2 label cluster app.kubernetes.io/instance=greptime-jzuuke Success `kbcli cluster label case.name=kbcli.test1 -l app.kubernetes.io/instance=greptime-jzuuke --namespace ns-oxzrz ` `kbcli cluster label greptime-jzuuke --list --namespace ns-oxzrz ` NAME NAMESPACE LABELS greptime-jzuuke ns-oxzrz app.kubernetes.io/instance=greptime-jzuuke case.name=kbcli.test1 clusterdefinition.kubeblocks.io/name=greptimedb clusterversion.kubeblocks.io/name=greptimedb-0.3.2 label cluster case.name=kbcli.test1 Success `kbcli cluster label greptime-jzuuke case.name=kbcli.test2 --overwrite --namespace ns-oxzrz ` `kbcli cluster label greptime-jzuuke --list --namespace ns-oxzrz ` NAME NAMESPACE LABELS greptime-jzuuke ns-oxzrz app.kubernetes.io/instance=greptime-jzuuke case.name=kbcli.test2 clusterdefinition.kubeblocks.io/name=greptimedb clusterversion.kubeblocks.io/name=greptimedb-0.3.2 label cluster case.name=kbcli.test2 Success `kbcli cluster label greptime-jzuuke case.name- --namespace ns-oxzrz ` `kbcli cluster label greptime-jzuuke --list --namespace ns-oxzrz ` NAME NAMESPACE LABELS greptime-jzuuke ns-oxzrz app.kubernetes.io/instance=greptime-jzuuke clusterdefinition.kubeblocks.io/name=greptimedb clusterversion.kubeblocks.io/name=greptimedb-0.3.2 delete cluster label case.name Success cluster connect `echo 'echo "show databases;" | greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash ` Defaulted container "frontend" out of: frontend, wait-datanode (init) Unable to use a TTY - input is not a terminal or the right kind of file 2025-09-01T03:24:58.809935Z INFO greptime: short_version: 0.3.2, full_version: greptimedb-HEAD-4b580f4 2025-09-01T03:24:58.809988Z INFO greptime: command line arguments 2025-09-01T03:24:58.809993Z INFO greptime: argument: greptime 2025-09-01T03:24:58.810001Z INFO greptime: argument: cli 2025-09-01T03:24:58.810004Z INFO greptime: argument: attach 2025-09-01T03:24:58.810008Z INFO greptime: argument: --grpc-addr 2025-09-01T03:24:58.810011Z INFO greptime: argument: greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001 Ready for commands. (Hint: try 'help') +---------+ | Schemas | +---------+ | public | +---------+ Total Rows: 1 Cost 148 ms connect cluster Success check component frontend exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=frontend --namespace ns-oxzrz | (grep "frontend" || true )` check component etcd exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=etcd --namespace ns-oxzrz | (grep "etcd" || true )` cluster vscale check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale greptime-jzuuke --auto-approve --force=true --components frontend,etcd --cpu 200m --memory 0.6Gi --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-verticalscaling-nkkbl created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-verticalscaling-nkkbl -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-verticalscaling-nkkbl ns-oxzrz VerticalScaling greptime-jzuuke frontend,etcd Creating -/- Sep 01,2025 11:24 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Updating Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating [Error] check cluster status timeout --------------------------------------get cluster greptime-jzuuke yaml-------------------------------------- `kubectl get cluster greptime-jzuuke -o yaml --namespace ns-oxzrz ` apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: annotations: kubeblocks.io/ops-request: '[***"name":"greptime-jzuuke-verticalscaling-nkkbl","type":"VerticalScaling"***]' kubeblocks.io/reconcile: "2025-09-01T03:31:03.807700254Z" kubectl.kubernetes.io/last-applied-configuration: | ***"apiVersion":"apps.kubeblocks.io/v1alpha1","kind":"Cluster","metadata":***"annotations":***,"name":"greptime-jzuuke","namespace":"ns-oxzrz"***,"spec":***"clusterDefinitionRef":"greptimedb","clusterVersionRef":"greptimedb-0.3.2","componentSpecs":[***"componentDefRef":"frontend","name":"frontend","replicas":2,"resources":***"limits":***"cpu":"100m","memory":"0.5Gi"***,"requests":***"cpu":"100m","memory":"0.5Gi"***,***"componentDefRef":"datanode","name":"datanode","replicas":2,"resources":***"limits":***"cpu":"100m","memory":"0.5Gi"***,"requests":***"cpu":"100m","memory":"0.5Gi"***,"volumeClaimTemplates":[***"name":"datanode","spec":***"accessModes":["ReadWriteOnce"],"resources":***"requests":***"storage":"1Gi"***,"storageClassName":null***]***,***"componentDefRef":"meta","name":"meta","replicas":1,"resources":***"limits":***"cpu":"100m","memory":"0.5Gi"***,"requests":***"cpu":"100m","memory":"0.5Gi"***,***"componentDefRef":"etcd","name":"etcd","replicas":3,"resources":***"limits":***"cpu":"100m","memory":"0.5Gi"***,"requests":***"cpu":"100m","memory":"0.5Gi"***,"volumeClaimTemplates":[***"name":"etcd-storage","spec":***"accessModes":["ReadWriteOnce"],"resources":***"requests":***"storage":"1Gi"***,"storageClassName":null***]***],"terminationPolicy":"WipeOut"*** creationTimestamp: "2025-09-01T03:18:09Z" finalizers: - cluster.kubeblocks.io/finalizer generation: 3 labels: app.kubernetes.io/instance: greptime-jzuuke clusterdefinition.kubeblocks.io/name: greptimedb clusterversion.kubeblocks.io/name: greptimedb-0.3.2 name: greptime-jzuuke namespace: ns-oxzrz resourceVersion: "26380" uid: b2d26b74-96f8-4648-817e-be200940db8a spec: clusterDefinitionRef: greptimedb clusterVersionRef: greptimedb-0.3.2 componentSpecs: - componentDefRef: frontend name: frontend replicas: 2 resources: limits: cpu: 200m memory: 644245094400m requests: cpu: 200m memory: 644245094400m - componentDefRef: datanode name: datanode replicas: 2 resources: limits: cpu: 100m memory: 512Mi requests: cpu: 100m memory: 512Mi volumeClaimTemplates: - name: datanode spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi - componentDefRef: meta name: meta replicas: 1 resources: limits: cpu: 100m memory: 512Mi requests: cpu: 100m memory: 512Mi - componentDefRef: etcd name: etcd replicas: 3 resources: limits: cpu: 200m memory: 644245094400m requests: cpu: 200m memory: 644245094400m volumeClaimTemplates: - name: etcd-storage spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi resources: cpu: "0" memory: "0" storage: size: "0" terminationPolicy: WipeOut status: clusterDefGeneration: 2 components: datanode: phase: Running podsReady: true podsReadyTime: "2025-09-01T03:24:41Z" etcd: phase: Updating podsReady: false podsReadyTime: "2025-09-01T03:21:13Z" frontend: phase: Running podsReady: true podsReadyTime: "2025-09-01T03:26:35Z" meta: phase: Running podsReady: true podsReadyTime: "2025-09-01T03:21:08Z" conditions: - lastTransitionTime: "2025-09-01T03:18:09Z" message: 'The operator has started the provisioning of Cluster: greptime-jzuuke' observedGeneration: 3 reason: PreCheckSucceed status: "True" type: ProvisioningStarted - lastTransitionTime: "2025-09-01T03:18:09Z" message: Successfully applied for resources observedGeneration: 3 reason: ApplyResourcesSucceed status: "True" type: ApplyResources - lastTransitionTime: "2025-09-01T03:25:00Z" message: 'pods are not ready in Components: [etcd], refer to related component message in Cluster.status.components' reason: ReplicasNotReady status: "False" type: ReplicasReady - lastTransitionTime: "2025-09-01T03:25:00Z" message: 'pods are unavailable in Components: [etcd], refer to related component message in Cluster.status.components' reason: ComponentsNotReady status: "False" type: Ready observedGeneration: 3 phase: Updating ------------------------------------------------------------------------------------------------------------------ --------------------------------------describe cluster greptime-jzuuke-------------------------------------- `kubectl describe cluster greptime-jzuuke --namespace ns-oxzrz ` Name: greptime-jzuuke Namespace: ns-oxzrz Labels: app.kubernetes.io/instance=greptime-jzuuke clusterdefinition.kubeblocks.io/name=greptimedb clusterversion.kubeblocks.io/name=greptimedb-0.3.2 Annotations: kubeblocks.io/ops-request: [***"name":"greptime-jzuuke-verticalscaling-nkkbl","type":"VerticalScaling"***] kubeblocks.io/reconcile: 2025-09-01T03:31:03.807700254Z API Version: apps.kubeblocks.io/v1alpha1 Kind: Cluster Metadata: Creation Timestamp: 2025-09-01T03:18:09Z Finalizers: cluster.kubeblocks.io/finalizer Generation: 3 Resource Version: 26380 UID: b2d26b74-96f8-4648-817e-be200940db8a Spec: Cluster Definition Ref: greptimedb Cluster Version Ref: greptimedb-0.3.2 Component Specs: Component Def Ref: frontend Name: frontend Replicas: 2 Resources: Limits: Cpu: 200m Memory: 644245094400m Requests: Cpu: 200m Memory: 644245094400m Component Def Ref: datanode Name: datanode Replicas: 2 Resources: Limits: Cpu: 100m Memory: 512Mi Requests: Cpu: 100m Memory: 512Mi Volume Claim Templates: Name: datanode Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 1Gi Component Def Ref: meta Name: meta Replicas: 1 Resources: Limits: Cpu: 100m Memory: 512Mi Requests: Cpu: 100m Memory: 512Mi Component Def Ref: etcd Name: etcd Replicas: 3 Resources: Limits: Cpu: 200m Memory: 644245094400m Requests: Cpu: 200m Memory: 644245094400m Volume Claim Templates: Name: etcd-storage Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 1Gi Resources: Cpu: 0 Memory: 0 Storage: Size: 0 Termination Policy: WipeOut Status: Cluster Def Generation: 2 Components: Datanode: Phase: Running Pods Ready: true Pods Ready Time: 2025-09-01T03:24:41Z Etcd: Phase: Updating Pods Ready: false Pods Ready Time: 2025-09-01T03:21:13Z Frontend: Phase: Running Pods Ready: true Pods Ready Time: 2025-09-01T03:26:35Z Meta: Phase: Running Pods Ready: true Pods Ready Time: 2025-09-01T03:21:08Z Conditions: Last Transition Time: 2025-09-01T03:18:09Z Message: The operator has started the provisioning of Cluster: greptime-jzuuke Observed Generation: 3 Reason: PreCheckSucceed Status: True Type: ProvisioningStarted Last Transition Time: 2025-09-01T03:18:09Z Message: Successfully applied for resources Observed Generation: 3 Reason: ApplyResourcesSucceed Status: True Type: ApplyResources Last Transition Time: 2025-09-01T03:25:00Z Message: pods are not ready in Components: [etcd], refer to related component message in Cluster.status.components Reason: ReplicasNotReady Status: False Type: ReplicasReady Last Transition Time: 2025-09-01T03:25:00Z Message: pods are unavailable in Components: [etcd], refer to related component message in Cluster.status.components Reason: ComponentsNotReady Status: False Type: Ready Observed Generation: 3 Phase: Updating Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ComponentPhaseTransition 14m (x4 over 14m) cluster-controller component is Creating Warning ComponentsNotReady 11m cluster-controller pods are unavailable in Components: [datanode etcd frontend], refer to related component message in Cluster.status.components Warning ReplicasNotReady 11m cluster-controller pods are not ready in Components: [datanode etcd frontend], refer to related component message in Cluster.status.components Warning ComponentsNotReady 11m cluster-controller pods are unavailable in Components: [datanode frontend], refer to related component message in Cluster.status.components Warning ReplicasNotReady 11m cluster-controller pods are not ready in Components: [datanode frontend], refer to related component message in Cluster.status.components Warning ReplicasNotReady 7m54s cluster-controller pods are not ready in Components: [datanode], refer to related component message in Cluster.status.components Warning ComponentsNotReady 7m54s cluster-controller pods are unavailable in Components: [datanode], refer to related component message in Cluster.status.components Normal ComponentPhaseTransition 7m49s (x4 over 11m) cluster-controller component is Running Normal AllReplicasReady 7m48s cluster-controller all pods of components are ready, waiting for the probe detection successful Normal ClusterReady 7m48s cluster-controller Cluster: greptime-jzuuke is ready, current phase is Running Normal Running 7m48s cluster-controller Cluster: greptime-jzuuke is ready, current phase is Running Normal ApplyResourcesSucceed 7m30s (x3 over 14m) cluster-controller Successfully applied for resources Normal PreCheckSucceed 7m30s (x3 over 14m) cluster-controller The operator has started the provisioning of Cluster: greptime-jzuuke Normal ComponentPhaseTransition 7m30s cluster-controller component is Updating Warning ReplicasNotReady 7m30s cluster-controller pods are not ready in Components: [etcd], refer to related component message in Cluster.status.components Warning ComponentsNotReady 7m30s cluster-controller pods are unavailable in Components: [etcd], refer to related component message in Cluster.status.components Warning FailedAttachVolume 87s event-controller Pod greptime-jzuuke-etcd-1: AttachVolume.Attach failed for volume "pvc-a4bdab19-e0e2-417d-91e4-523765a7a353" : timed out waiting for external-attacher of disk.csi.azure.com CSI driver to attach volume /subscriptions/e659c16c-8ba9-41ab-98a0-d64b0237ba45/resourceGroups/MC_cicd-aks-otzy9s9z-group_cicd-aks-otzy9s9z_eastus/providers/Microsoft.Compute/disks/pvc-a4bdab19-e0e2-417d-91e4-523765a7a353 ------------------------------------------------------------------------------------------------------------------ check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 100m / 100m 512Mi / 512Mi datanode:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 100m / 100m 512Mi / 512Mi datanode:1Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:21 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running leader 0 100m / 100m 512Mi / 512Mi etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Init:0/1 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:26 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:25 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:26 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:25 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 check pod status done check cluster status again check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-verticalscaling-nkkbl ns-oxzrz VerticalScaling greptime-jzuuke frontend,etcd Running 3/5 Sep 01,2025 11:24 UTC+0800 ops_status:greptime-jzuuke-verticalscaling-nkkbl ns-oxzrz VerticalScaling greptime-jzuuke frontend,etcd Running 3/5 Sep 01,2025 11:24 UTC+0800 ops_status:greptime-jzuuke-verticalscaling-nkkbl ns-oxzrz VerticalScaling greptime-jzuuke frontend,etcd Running 3/5 Sep 01,2025 11:24 UTC+0800 ops_status:greptime-jzuuke-verticalscaling-nkkbl ns-oxzrz VerticalScaling greptime-jzuuke frontend,etcd Running 4/5 Sep 01,2025 11:24 UTC+0800 ops_status:greptime-jzuuke-verticalscaling-nkkbl ns-oxzrz VerticalScaling greptime-jzuuke frontend,etcd Running 4/5 Sep 01,2025 11:24 UTC+0800 ops_status:greptime-jzuuke-verticalscaling-nkkbl ns-oxzrz VerticalScaling greptime-jzuuke frontend,etcd Running 4/5 Sep 01,2025 11:24 UTC+0800 check ops status done ops_status:greptime-jzuuke-verticalscaling-nkkbl ns-oxzrz VerticalScaling greptime-jzuuke frontend,etcd Succeed 5/5 Sep 01,2025 11:24 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-verticalscaling-nkkbl --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-verticalscaling-nkkbl patched `kbcli cluster delete-ops --name greptime-jzuuke-verticalscaling-nkkbl --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-verticalscaling-nkkbl deleted check component frontend exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=frontend --namespace ns-oxzrz | (grep "frontend" || true )` check component etcd exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=etcd --namespace ns-oxzrz | (grep "etcd" || true )` cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart greptime-jzuuke --auto-approve --force=true --components frontend,etcd --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-restart-nc2zk created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-restart-nc2zk -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-restart-nc2zk ns-oxzrz Restart greptime-jzuuke frontend,etcd Creating -/- Sep 01,2025 11:35 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Updating Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating [Error] check cluster status timeout --------------------------------------get cluster greptime-jzuuke yaml-------------------------------------- `kubectl get cluster greptime-jzuuke -o yaml --namespace ns-oxzrz ` apiVersion: apps.kubeblocks.io/v1alpha1 kind: Cluster metadata: annotations: kubeblocks.io/ops-request: '[***"name":"greptime-jzuuke-restart-nc2zk","type":"Restart"***]' kubeblocks.io/reconcile: "2025-09-01T03:35:06.94240972Z" kubectl.kubernetes.io/last-applied-configuration: | ***"apiVersion":"apps.kubeblocks.io/v1alpha1","kind":"Cluster","metadata":***"annotations":***,"name":"greptime-jzuuke","namespace":"ns-oxzrz"***,"spec":***"clusterDefinitionRef":"greptimedb","clusterVersionRef":"greptimedb-0.3.2","componentSpecs":[***"componentDefRef":"frontend","name":"frontend","replicas":2,"resources":***"limits":***"cpu":"100m","memory":"0.5Gi"***,"requests":***"cpu":"100m","memory":"0.5Gi"***,***"componentDefRef":"datanode","name":"datanode","replicas":2,"resources":***"limits":***"cpu":"100m","memory":"0.5Gi"***,"requests":***"cpu":"100m","memory":"0.5Gi"***,"volumeClaimTemplates":[***"name":"datanode","spec":***"accessModes":["ReadWriteOnce"],"resources":***"requests":***"storage":"1Gi"***,"storageClassName":null***]***,***"componentDefRef":"meta","name":"meta","replicas":1,"resources":***"limits":***"cpu":"100m","memory":"0.5Gi"***,"requests":***"cpu":"100m","memory":"0.5Gi"***,***"componentDefRef":"etcd","name":"etcd","replicas":3,"resources":***"limits":***"cpu":"100m","memory":"0.5Gi"***,"requests":***"cpu":"100m","memory":"0.5Gi"***,"volumeClaimTemplates":[***"name":"etcd-storage","spec":***"accessModes":["ReadWriteOnce"],"resources":***"requests":***"storage":"1Gi"***,"storageClassName":null***]***],"terminationPolicy":"WipeOut"*** creationTimestamp: "2025-09-01T03:18:09Z" finalizers: - cluster.kubeblocks.io/finalizer generation: 3 labels: app.kubernetes.io/instance: greptime-jzuuke clusterdefinition.kubeblocks.io/name: greptimedb clusterversion.kubeblocks.io/name: greptimedb-0.3.2 name: greptime-jzuuke namespace: ns-oxzrz resourceVersion: "30854" uid: b2d26b74-96f8-4648-817e-be200940db8a spec: clusterDefinitionRef: greptimedb clusterVersionRef: greptimedb-0.3.2 componentSpecs: - componentDefRef: frontend name: frontend replicas: 2 resources: limits: cpu: 200m memory: 644245094400m requests: cpu: 200m memory: 644245094400m - componentDefRef: datanode name: datanode replicas: 2 resources: limits: cpu: 100m memory: 512Mi requests: cpu: 100m memory: 512Mi volumeClaimTemplates: - name: datanode spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi - componentDefRef: meta name: meta replicas: 1 resources: limits: cpu: 100m memory: 512Mi requests: cpu: 100m memory: 512Mi - componentDefRef: etcd name: etcd replicas: 3 resources: limits: cpu: 200m memory: 644245094400m requests: cpu: 200m memory: 644245094400m volumeClaimTemplates: - name: etcd-storage spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi resources: cpu: "0" memory: "0" storage: size: "0" terminationPolicy: WipeOut status: clusterDefGeneration: 2 components: datanode: phase: Running podsReady: true podsReadyTime: "2025-09-01T03:24:41Z" etcd: phase: Updating podsReady: false podsReadyTime: "2025-09-01T03:35:48Z" frontend: phase: Running podsReady: true podsReadyTime: "2025-09-01T03:36:22Z" meta: phase: Running podsReady: true podsReadyTime: "2025-09-01T03:21:08Z" conditions: - lastTransitionTime: "2025-09-01T03:18:09Z" message: 'The operator has started the provisioning of Cluster: greptime-jzuuke' observedGeneration: 3 reason: PreCheckSucceed status: "True" type: ProvisioningStarted - lastTransitionTime: "2025-09-01T03:18:09Z" message: Successfully applied for resources observedGeneration: 3 reason: ApplyResourcesSucceed status: "True" type: ApplyResources - lastTransitionTime: "2025-09-01T03:35:53Z" message: 'pods are not ready in Components: [etcd], refer to related component message in Cluster.status.components' reason: ReplicasNotReady status: "False" type: ReplicasReady - lastTransitionTime: "2025-09-01T03:35:53Z" message: 'pods are unavailable in Components: [etcd], refer to related component message in Cluster.status.components' reason: ComponentsNotReady status: "False" type: Ready observedGeneration: 3 phase: Updating ------------------------------------------------------------------------------------------------------------------ --------------------------------------describe cluster greptime-jzuuke-------------------------------------- `kubectl describe cluster greptime-jzuuke --namespace ns-oxzrz ` Name: greptime-jzuuke Namespace: ns-oxzrz Labels: app.kubernetes.io/instance=greptime-jzuuke clusterdefinition.kubeblocks.io/name=greptimedb clusterversion.kubeblocks.io/name=greptimedb-0.3.2 Annotations: kubeblocks.io/ops-request: [***"name":"greptime-jzuuke-restart-nc2zk","type":"Restart"***] kubeblocks.io/reconcile: 2025-09-01T03:35:06.94240972Z API Version: apps.kubeblocks.io/v1alpha1 Kind: Cluster Metadata: Creation Timestamp: 2025-09-01T03:18:09Z Finalizers: cluster.kubeblocks.io/finalizer Generation: 3 Resource Version: 30854 UID: b2d26b74-96f8-4648-817e-be200940db8a Spec: Cluster Definition Ref: greptimedb Cluster Version Ref: greptimedb-0.3.2 Component Specs: Component Def Ref: frontend Name: frontend Replicas: 2 Resources: Limits: Cpu: 200m Memory: 644245094400m Requests: Cpu: 200m Memory: 644245094400m Component Def Ref: datanode Name: datanode Replicas: 2 Resources: Limits: Cpu: 100m Memory: 512Mi Requests: Cpu: 100m Memory: 512Mi Volume Claim Templates: Name: datanode Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 1Gi Component Def Ref: meta Name: meta Replicas: 1 Resources: Limits: Cpu: 100m Memory: 512Mi Requests: Cpu: 100m Memory: 512Mi Component Def Ref: etcd Name: etcd Replicas: 3 Resources: Limits: Cpu: 200m Memory: 644245094400m Requests: Cpu: 200m Memory: 644245094400m Volume Claim Templates: Name: etcd-storage Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 1Gi Resources: Cpu: 0 Memory: 0 Storage: Size: 0 Termination Policy: WipeOut Status: Cluster Def Generation: 2 Components: Datanode: Phase: Running Pods Ready: true Pods Ready Time: 2025-09-01T03:24:41Z Etcd: Phase: Updating Pods Ready: false Pods Ready Time: 2025-09-01T03:35:48Z Frontend: Phase: Running Pods Ready: true Pods Ready Time: 2025-09-01T03:36:22Z Meta: Phase: Running Pods Ready: true Pods Ready Time: 2025-09-01T03:21:08Z Conditions: Last Transition Time: 2025-09-01T03:18:09Z Message: The operator has started the provisioning of Cluster: greptime-jzuuke Observed Generation: 3 Reason: PreCheckSucceed Status: True Type: ProvisioningStarted Last Transition Time: 2025-09-01T03:18:09Z Message: Successfully applied for resources Observed Generation: 3 Reason: ApplyResourcesSucceed Status: True Type: ApplyResources Last Transition Time: 2025-09-01T03:35:53Z Message: pods are not ready in Components: [etcd], refer to related component message in Cluster.status.components Reason: ReplicasNotReady Status: False Type: ReplicasReady Last Transition Time: 2025-09-01T03:35:53Z Message: pods are unavailable in Components: [etcd], refer to related component message in Cluster.status.components Reason: ComponentsNotReady Status: False Type: Ready Observed Generation: 3 Phase: Updating Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ComponentPhaseTransition 25m (x4 over 25m) cluster-controller component is Creating Warning ComponentsNotReady 22m cluster-controller pods are unavailable in Components: [datanode etcd frontend], refer to related component message in Cluster.status.components Warning ReplicasNotReady 22m cluster-controller pods are not ready in Components: [datanode etcd frontend], refer to related component message in Cluster.status.components Warning ComponentsNotReady 22m cluster-controller pods are unavailable in Components: [datanode frontend], refer to related component message in Cluster.status.components Warning ReplicasNotReady 22m cluster-controller pods are not ready in Components: [datanode frontend], refer to related component message in Cluster.status.components Warning ReplicasNotReady 18m cluster-controller pods are not ready in Components: [datanode], refer to related component message in Cluster.status.components Warning ComponentsNotReady 18m cluster-controller pods are unavailable in Components: [datanode], refer to related component message in Cluster.status.components Normal ClusterReady 18m cluster-controller Cluster: greptime-jzuuke is ready, current phase is Running Normal Running 18m cluster-controller Cluster: greptime-jzuuke is ready, current phase is Running Normal ApplyResourcesSucceed 18m (x3 over 25m) cluster-controller Successfully applied for resources Normal PreCheckSucceed 18m (x3 over 25m) cluster-controller The operator has started the provisioning of Cluster: greptime-jzuuke Normal ComponentPhaseTransition 18m cluster-controller component is Updating Warning ReplicasNotReady 18m cluster-controller pods are not ready in Components: [etcd], refer to related component message in Cluster.status.components Warning ComponentsNotReady 18m cluster-controller pods are unavailable in Components: [etcd], refer to related component message in Cluster.status.components Warning FailedAttachVolume 8m18s (x3 over 12m) event-controller Pod greptime-jzuuke-etcd-1: AttachVolume.Attach failed for volume "pvc-a4bdab19-e0e2-417d-91e4-523765a7a353" : timed out waiting for external-attacher of disk.csi.azure.com CSI driver to attach volume /subscriptions/e659c16c-8ba9-41ab-98a0-d64b0237ba45/resourceGroups/MC_cicd-aks-otzy9s9z-group_cicd-aks-otzy9s9z_eastus/providers/Microsoft.Compute/disks/pvc-a4bdab19-e0e2-417d-91e4-523765a7a353 Normal ComponentPhaseTransition 7m36s (x6 over 22m) cluster-controller component is Running Normal AllReplicasReady 7m36s (x2 over 18m) cluster-controller all pods of components are ready, waiting for the probe detection successful ------------------------------------------------------------------------------------------------------------------ check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 100m / 100m 512Mi / 512Mi datanode:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 100m / 100m 512Mi / 512Mi datanode:1Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:21 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Init:0/1 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:38 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:36 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running leader 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:35 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:36 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:36 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:18 UTC+0800 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:Init:0/1 pod_status:PodInitializing pod_status:PodInitializing check pod status done check cluster status again check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-restart-nc2zk ns-oxzrz Restart greptime-jzuuke frontend,etcd Running 4/5 Sep 01,2025 11:35 UTC+0800 ops_status:greptime-jzuuke-restart-nc2zk ns-oxzrz Restart greptime-jzuuke frontend,etcd Running 4/5 Sep 01,2025 11:35 UTC+0800 check ops status done ops_status:greptime-jzuuke-restart-nc2zk ns-oxzrz Restart greptime-jzuuke frontend,etcd Succeed 5/5 Sep 01,2025 11:35 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-restart-nc2zk --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-restart-nc2zk patched `kbcli cluster delete-ops --name greptime-jzuuke-restart-nc2zk --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-restart-nc2zk deleted cluster stop check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster stop greptime-jzuuke --auto-approve --force=true --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-stop-4tzb8 created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-stop-4tzb8 -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-stop-4tzb8 ns-oxzrz Stop greptime-jzuuke Creating -/- Sep 01,2025 11:44 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Stopping Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Stopping cluster_status:Stopping cluster_status:Stopping check cluster status done cluster_status:Stopped check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME check pod status done check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-stop-4tzb8 ns-oxzrz Stop greptime-jzuuke datanode,etcd,frontend,meta Succeed 8/8 Sep 01,2025 11:44 UTC+0800 check ops status done ops_status:greptime-jzuuke-stop-4tzb8 ns-oxzrz Stop greptime-jzuuke datanode,etcd,frontend,meta Succeed 8/8 Sep 01,2025 11:44 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-stop-4tzb8 --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-stop-4tzb8 patched `kbcli cluster delete-ops --name greptime-jzuuke-stop-4tzb8 --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-stop-4tzb8 deleted cluster start check cluster status before ops check cluster status done cluster_status:Stopped `kbcli cluster start greptime-jzuuke --force=true --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-start-cwb54 created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-start-cwb54 -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-start-cwb54 ns-oxzrz Start greptime-jzuuke Creating -/- Sep 01,2025 11:44 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Updating Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 100m / 100m 512Mi / 512Mi datanode:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:44 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 100m / 100m 512Mi / 512Mi datanode:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:45 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running leader 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:44 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:44 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:44 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:44 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:45 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:44 UTC+0800 check pod status done check cluster connect `echo 'greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash` check cluster connect done check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-start-cwb54 ns-oxzrz Start greptime-jzuuke datanode,etcd,frontend,meta Succeed 8/8 Sep 01,2025 11:44 UTC+0800 check ops status done ops_status:greptime-jzuuke-start-cwb54 ns-oxzrz Start greptime-jzuuke datanode,etcd,frontend,meta Succeed 8/8 Sep 01,2025 11:44 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-start-cwb54 --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-start-cwb54 patched `kbcli cluster delete-ops --name greptime-jzuuke-start-cwb54 --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-start-cwb54 deleted check component meta exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=meta --namespace ns-oxzrz | (grep "meta" || true )` cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart greptime-jzuuke --auto-approve --force=true --components meta --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-restart-8wbld created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-restart-8wbld -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-restart-8wbld ns-oxzrz Restart greptime-jzuuke meta Creating -/- Sep 01,2025 11:51 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Updating Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 100m / 100m 512Mi / 512Mi datanode:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:44 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 100m / 100m 512Mi / 512Mi datanode:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:45 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running leader 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:44 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:44 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:44 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:44 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:45 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:51 UTC+0800 check pod status done check cluster connect `echo 'greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash` check cluster connect done check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-restart-8wbld ns-oxzrz Restart greptime-jzuuke meta Succeed 1/1 Sep 01,2025 11:51 UTC+0800 check ops status done ops_status:greptime-jzuuke-restart-8wbld ns-oxzrz Restart greptime-jzuuke meta Succeed 1/1 Sep 01,2025 11:51 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-restart-8wbld --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-restart-8wbld patched `kbcli cluster delete-ops --name greptime-jzuuke-restart-8wbld --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-restart-8wbld deleted cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart greptime-jzuuke --auto-approve --force=true --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-restart-trndx created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-restart-trndx -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-restart-trndx ns-oxzrz Restart greptime-jzuuke frontend,datanode,meta,etcd Creating -/- Sep 01,2025 11:52 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Updating Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 100m / 100m 512Mi / 512Mi datanode:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 100m / 100m 512Mi / 512Mi datanode:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:53 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running leader 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:52 UTC+0800 check pod status done check cluster connect `echo 'greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash` check cluster connect done check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-restart-trndx ns-oxzrz Restart greptime-jzuuke frontend,datanode,meta,etcd Succeed 8/8 Sep 01,2025 11:52 UTC+0800 check ops status done ops_status:greptime-jzuuke-restart-trndx ns-oxzrz Restart greptime-jzuuke frontend,datanode,meta,etcd Succeed 8/8 Sep 01,2025 11:52 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-restart-trndx --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-restart-trndx patched `kbcli cluster delete-ops --name greptime-jzuuke-restart-trndx --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-restart-trndx deleted check component datanode exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=datanode --namespace ns-oxzrz | (grep "datanode" || true )` cluster vscale check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale greptime-jzuuke --auto-approve --force=true --components datanode --cpu 200m --memory 0.6Gi --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-verticalscaling-rlq5j created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-verticalscaling-rlq5j -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-verticalscaling-rlq5j ns-oxzrz VerticalScaling greptime-jzuuke datanode Creating -/- Sep 01,2025 11:53 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Updating Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:53 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:53 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running leader 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 100m / 100m 512Mi / 512Mi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:52 UTC+0800 check pod status done check cluster connect `echo 'greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash` check cluster connect done check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-verticalscaling-rlq5j ns-oxzrz VerticalScaling greptime-jzuuke datanode Succeed 2/2 Sep 01,2025 11:53 UTC+0800 check ops status done ops_status:greptime-jzuuke-verticalscaling-rlq5j ns-oxzrz VerticalScaling greptime-jzuuke datanode Succeed 2/2 Sep 01,2025 11:53 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-verticalscaling-rlq5j --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-verticalscaling-rlq5j patched `kbcli cluster delete-ops --name greptime-jzuuke-verticalscaling-rlq5j --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-verticalscaling-rlq5j deleted check component meta exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=meta --namespace ns-oxzrz | (grep "meta" || true )` cluster vscale check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster vscale greptime-jzuuke --auto-approve --force=true --components meta --cpu 200m --memory 0.6Gi --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-verticalscaling-jxdz5 created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-verticalscaling-jxdz5 -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-verticalscaling-jxdz5 ns-oxzrz VerticalScaling greptime-jzuuke meta Creating -/- Sep 01,2025 11:54 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Updating Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:53 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:53 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running leader 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000004/10.224.0.6 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 check pod status done check cluster connect `echo 'greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash` check cluster connect done check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-verticalscaling-jxdz5 ns-oxzrz VerticalScaling greptime-jzuuke meta Succeed 1/1 Sep 01,2025 11:54 UTC+0800 check ops status done ops_status:greptime-jzuuke-verticalscaling-jxdz5 ns-oxzrz VerticalScaling greptime-jzuuke meta Succeed 1/1 Sep 01,2025 11:54 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-verticalscaling-jxdz5 --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-verticalscaling-jxdz5 patched `kbcli cluster delete-ops --name greptime-jzuuke-verticalscaling-jxdz5 --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-verticalscaling-jxdz5 deleted check component datanode exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=datanode --namespace ns-oxzrz | (grep "datanode" || true )` `kubectl get pvc -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=datanode,apps.kubeblocks.io/vct-name=datanode --namespace ns-oxzrz ` cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in greptime-jzuuke namespace. `kbcli cluster volume-expand greptime-jzuuke --auto-approve --force=true --components datanode --volume-claim-templates datanode --storage 5Gi --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-volumeexpansion-fvv56 created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-volumeexpansion-fvv56 -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-volumeexpansion-fvv56 ns-oxzrz VolumeExpansion greptime-jzuuke datanode Creating -/- Sep 01,2025 11:54 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Updating Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:53 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running leader 0 200m / 200m 644245094400m / 644245094400m etcd-storage:1Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 check pod status done check cluster connect `echo 'greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash` check cluster connect done No resources found in greptime-jzuuke namespace. check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-volumeexpansion-fvv56 ns-oxzrz VolumeExpansion greptime-jzuuke datanode Succeed 2/2 Sep 01,2025 11:54 UTC+0800 check ops status done ops_status:greptime-jzuuke-volumeexpansion-fvv56 ns-oxzrz VolumeExpansion greptime-jzuuke datanode Succeed 2/2 Sep 01,2025 11:54 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-volumeexpansion-fvv56 --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-volumeexpansion-fvv56 patched `kbcli cluster delete-ops --name greptime-jzuuke-volumeexpansion-fvv56 --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-volumeexpansion-fvv56 deleted check component etcd exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=etcd --namespace ns-oxzrz | (grep "etcd" || true )` `kubectl get pvc -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=etcd,apps.kubeblocks.io/vct-name=etcd-storage --namespace ns-oxzrz ` cluster volume-expand check cluster status before ops check cluster status done cluster_status:Running No resources found in greptime-jzuuke namespace. `kbcli cluster volume-expand greptime-jzuuke --auto-approve --force=true --components etcd --volume-claim-templates etcd-storage --storage 6Gi --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-volumeexpansion-546t6 created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-volumeexpansion-546t6 -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-volumeexpansion-546t6 ns-oxzrz VolumeExpansion greptime-jzuuke etcd Pending -/- Sep 01,2025 12:06 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Updating Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:53 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running leader 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 check pod status done check cluster connect `echo 'greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash` check cluster connect done No resources found in greptime-jzuuke namespace. check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-volumeexpansion-546t6 ns-oxzrz VolumeExpansion greptime-jzuuke etcd Succeed 3/3 Sep 01,2025 12:06 UTC+0800 check ops status done ops_status:greptime-jzuuke-volumeexpansion-546t6 ns-oxzrz VolumeExpansion greptime-jzuuke etcd Succeed 3/3 Sep 01,2025 12:06 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-volumeexpansion-546t6 --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-volumeexpansion-546t6 patched `kbcli cluster delete-ops --name greptime-jzuuke-volumeexpansion-546t6 --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-volumeexpansion-546t6 deleted check component frontend exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=frontend --namespace ns-oxzrz | (grep "frontend" || true )` check component datanode exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=datanode --namespace ns-oxzrz | (grep "datanode" || true )` cluster hscale check cluster status before ops check cluster status done cluster_status:Running No resources found in greptime-jzuuke namespace. `kbcli cluster hscale greptime-jzuuke --auto-approve --force=true --components frontend,datanode --replicas 3 --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-horizontalscaling-4jh5q created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-horizontalscaling-4jh5q -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-horizontalscaling-4jh5q ns-oxzrz HorizontalScaling greptime-jzuuke frontend,datanode Running -/- Sep 01,2025 12:12 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Updating Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-datanode-2 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 12:12 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:53 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running leader 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-frontend-2 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 12:12 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 check pod status done check cluster connect `echo 'greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash` check cluster connect done No resources found in greptime-jzuuke namespace. check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-horizontalscaling-4jh5q ns-oxzrz HorizontalScaling greptime-jzuuke frontend,datanode Succeed 2/2 Sep 01,2025 12:12 UTC+0800 check ops status done ops_status:greptime-jzuuke-horizontalscaling-4jh5q ns-oxzrz HorizontalScaling greptime-jzuuke frontend,datanode Succeed 2/2 Sep 01,2025 12:12 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-horizontalscaling-4jh5q --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-horizontalscaling-4jh5q patched `kbcli cluster delete-ops --name greptime-jzuuke-horizontalscaling-4jh5q --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-horizontalscaling-4jh5q deleted check component frontend exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=frontend --namespace ns-oxzrz | (grep "frontend" || true )` check component datanode exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=datanode --namespace ns-oxzrz | (grep "datanode" || true )` cluster hscale check cluster status before ops check cluster status done cluster_status:Running No resources found in greptime-jzuuke namespace. `kbcli cluster hscale greptime-jzuuke --auto-approve --force=true --components frontend,datanode --replicas 2 --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-horizontalscaling-5nfl7 created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-horizontalscaling-5nfl7 -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-horizontalscaling-5nfl7 ns-oxzrz HorizontalScaling greptime-jzuuke frontend,datanode Creating -/- Sep 01,2025 12:12 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Updating Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:53 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running leader 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 check pod status done check cluster connect `echo 'greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash` check cluster connect done No resources found in greptime-jzuuke namespace. check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-horizontalscaling-5nfl7 ns-oxzrz HorizontalScaling greptime-jzuuke frontend,datanode Succeed 2/2 Sep 01,2025 12:12 UTC+0800 check ops status done ops_status:greptime-jzuuke-horizontalscaling-5nfl7 ns-oxzrz HorizontalScaling greptime-jzuuke frontend,datanode Succeed 2/2 Sep 01,2025 12:12 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-horizontalscaling-5nfl7 --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-horizontalscaling-5nfl7 patched `kbcli cluster delete-ops --name greptime-jzuuke-horizontalscaling-5nfl7 --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-horizontalscaling-5nfl7 deleted 13 check component datanode exists `kubectl get components -l app.kubernetes.io/instance=greptime-jzuuke,apps.kubeblocks.io/component-name=datanode --namespace ns-oxzrz | (grep "datanode" || true )` cluster restart check cluster status before ops check cluster status done cluster_status:Running `kbcli cluster restart greptime-jzuuke --auto-approve --force=true --components datanode --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-restart-stqz8 created successfully, you can view the progress: kbcli cluster describe-ops greptime-jzuuke-restart-stqz8 -n ns-oxzrz check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-restart-stqz8 ns-oxzrz Restart greptime-jzuuke datanode Creating -/- Sep 01,2025 12:13 UTC+0800 check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Updating Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating cluster_status:Updating check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 12:13 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 12:13 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:53 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running leader 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 check pod status done check cluster connect `echo 'greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash` check cluster connect done check ops status `kbcli cluster list-ops greptime-jzuuke --status all --namespace ns-oxzrz ` NAME NAMESPACE TYPE CLUSTER COMPONENT STATUS PROGRESS CREATED-TIME greptime-jzuuke-restart-stqz8 ns-oxzrz Restart greptime-jzuuke datanode Succeed 2/2 Sep 01,2025 12:13 UTC+0800 check ops status done ops_status:greptime-jzuuke-restart-stqz8 ns-oxzrz Restart greptime-jzuuke datanode Succeed 2/2 Sep 01,2025 12:13 UTC+0800 `kubectl patch -p '***"metadata":***"finalizers":[]***' --type=merge opsrequests greptime-jzuuke-restart-stqz8 --namespace ns-oxzrz ` opsrequest.apps.kubeblocks.io/greptime-jzuuke-restart-stqz8 patched `kbcli cluster delete-ops --name greptime-jzuuke-restart-stqz8 --force --auto-approve --namespace ns-oxzrz ` OpsRequest greptime-jzuuke-restart-stqz8 deleted cluster update terminationPolicy WipeOut `kbcli cluster update greptime-jzuuke --termination-policy=WipeOut --namespace ns-oxzrz ` cluster.apps.kubeblocks.io/greptime-jzuuke updated (no change) check cluster status `kbcli cluster list greptime-jzuuke --show-labels --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME LABELS greptime-jzuuke ns-oxzrz greptimedb greptimedb-0.3.2 WipeOut Running Sep 01,2025 11:18 UTC+0800 app.kubernetes.io/instance=greptime-jzuuke,clusterdefinition.kubeblocks.io/name=greptimedb,clusterversion.kubeblocks.io/name=greptimedb-0.3.2 check cluster status done cluster_status:Running check pod status `kbcli cluster list-instances greptime-jzuuke --namespace ns-oxzrz ` NAME NAMESPACE CLUSTER COMPONENT STATUS ROLE ACCESSMODE AZ CPU(REQUEST/LIMIT) MEMORY(REQUEST/LIMIT) STORAGE NODE CREATED-TIME greptime-jzuuke-datanode-0 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 12:13 UTC+0800 greptime-jzuuke-datanode-1 ns-oxzrz greptime-jzuuke datanode Running 0 200m / 200m 644245094400m / 644245094400m datanode:5Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 12:13 UTC+0800 greptime-jzuuke-etcd-0 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:53 UTC+0800 greptime-jzuuke-etcd-1 ns-oxzrz greptime-jzuuke etcd Running follower 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-etcd-2 ns-oxzrz greptime-jzuuke etcd Running leader 0 200m / 200m 644245094400m / 644245094400m etcd-storage:6Gi aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-0 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:52 UTC+0800 greptime-jzuuke-frontend-1 ns-oxzrz greptime-jzuuke frontend Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000006/10.224.0.10 Sep 01,2025 11:55 UTC+0800 greptime-jzuuke-meta-0 ns-oxzrz greptime-jzuuke meta Running 0 200m / 200m 644245094400m / 644245094400m aks-cicdamdpool-25950949-vmss000007/10.224.0.5 Sep 01,2025 11:54 UTC+0800 check pod status done check cluster connect `echo 'greptime cli attach --grpc-addr greptime-jzuuke-frontend.ns-oxzrz.svc.cluster.local:4001' | kubectl exec -it greptime-jzuuke-frontend-0 --namespace ns-oxzrz -- bash` check cluster connect done cluster list-logs `kbcli cluster list-logs greptime-jzuuke --namespace ns-oxzrz ` No log files found. You can enable the log feature with the kbcli command below. kbcli cluster update greptime-jzuuke --enable-all-logs=true --namespace ns-oxzrz Error from server (NotFound): pods "greptime-jzuuke-frontend-0" not found cluster logs `kbcli cluster logs greptime-jzuuke --tail 30 --namespace ns-oxzrz ` Defaulted container "etcd" out of: etcd, lorry, volume-permissions (init) ***"level":"warn","ts":"2025-09-01T04:01:46.794Z","caller":"etcdserver/cluster_util.go:288","msg":"failed to reach the peer URL","address":"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version","remote-member-id":"8047f29c1627a776","error":"Get \"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version\": dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:01:46.794Z","caller":"etcdserver/cluster_util.go:155","msg":"failed to get version","remote-member-id":"8047f29c1627a776","error":"Get \"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version\": dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:01:49.225Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"8047f29c1627a776","rtt":"4.715608ms","error":"dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:01:49.225Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"8047f29c1627a776","rtt":"555.85µs","error":"dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:01:50.800Z","caller":"etcdserver/cluster_util.go:288","msg":"failed to reach the peer URL","address":"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version","remote-member-id":"8047f29c1627a776","error":"Get \"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version\": dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:01:50.800Z","caller":"etcdserver/cluster_util.go:155","msg":"failed to get version","remote-member-id":"8047f29c1627a776","error":"Get \"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version\": dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:01:54.226Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"8047f29c1627a776","rtt":"4.715608ms","error":"dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:01:54.226Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"8047f29c1627a776","rtt":"555.85µs","error":"dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"info","ts":"2025-09-01T04:01:54.538Z","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"1c0b62c02f7c3f33","to":"8047f29c1627a776","stream-type":"stream Message"*** ***"level":"info","ts":"2025-09-01T04:01:54.538Z","caller":"rafthttp/peer_status.go:53","msg":"peer became active","peer-id":"8047f29c1627a776"*** ***"level":"info","ts":"2025-09-01T04:01:54.538Z","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"1c0b62c02f7c3f33","remote-peer-id":"8047f29c1627a776"*** ***"level":"info","ts":"2025-09-01T04:01:54.539Z","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"1c0b62c02f7c3f33","to":"8047f29c1627a776","stream-type":"stream MsgApp v2"*** ***"level":"warn","ts":"2025-09-01T04:01:54.539Z","caller":"rafthttp/stream.go:265","msg":"closed TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","local-member-id":"1c0b62c02f7c3f33","remote-peer-id":"8047f29c1627a776"*** ***"level":"info","ts":"2025-09-01T04:01:54.539Z","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","local-member-id":"1c0b62c02f7c3f33","remote-peer-id":"8047f29c1627a776"*** ***"level":"warn","ts":"2025-09-01T04:01:54.586Z","caller":"rafthttp/peer_status.go:66","msg":"peer became inactive (message send to peer failed)","peer-id":"8047f29c1627a776","error":"failed to dial 8047f29c1627a776 on stream MsgApp v2 (dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host)"*** ***"level":"warn","ts":"2025-09-01T04:01:54.806Z","caller":"etcdserver/cluster_util.go:288","msg":"failed to reach the peer URL","address":"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version","remote-member-id":"8047f29c1627a776","error":"Get \"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version\": dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:01:54.806Z","caller":"etcdserver/cluster_util.go:155","msg":"failed to get version","remote-member-id":"8047f29c1627a776","error":"Get \"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version\": dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:01:58.812Z","caller":"etcdserver/cluster_util.go:288","msg":"failed to reach the peer URL","address":"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version","remote-member-id":"8047f29c1627a776","error":"Get \"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version\": dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:01:58.812Z","caller":"etcdserver/cluster_util.go:155","msg":"failed to get version","remote-member-id":"8047f29c1627a776","error":"Get \"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version\": dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:01:59.226Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"8047f29c1627a776","rtt":"555.85µs","error":"dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:01:59.226Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"8047f29c1627a776","rtt":"4.715608ms","error":"dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:02:02.820Z","caller":"etcdserver/cluster_util.go:288","msg":"failed to reach the peer URL","address":"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version","remote-member-id":"8047f29c1627a776","error":"Get \"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version\": dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:02:02.820Z","caller":"etcdserver/cluster_util.go:155","msg":"failed to get version","remote-member-id":"8047f29c1627a776","error":"Get \"http://greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local:2380/version\": dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:02:04.227Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"8047f29c1627a776","rtt":"4.715608ms","error":"dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:02:04.227Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"8047f29c1627a776","rtt":"555.85µs","error":"dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"info","ts":"2025-09-01T04:02:04.579Z","caller":"rafthttp/peer_status.go:53","msg":"peer became active","peer-id":"8047f29c1627a776"*** ***"level":"info","ts":"2025-09-01T04:02:04.579Z","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"1c0b62c02f7c3f33","remote-peer-id":"8047f29c1627a776"*** ***"level":"info","ts":"2025-09-01T04:02:04.579Z","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream Message","local-member-id":"1c0b62c02f7c3f33","remote-peer-id":"8047f29c1627a776"*** ***"level":"warn","ts":"2025-09-01T04:02:09.228Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"8047f29c1627a776","rtt":"555.85µs","error":"dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** ***"level":"warn","ts":"2025-09-01T04:02:09.228Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"8047f29c1627a776","rtt":"4.715608ms","error":"dial tcp: lookup greptime-jzuuke-etcd-1.greptime-jzuuke-etcd-headless.ns-oxzrz.svc.cluster.local on 10.0.0.10:53: no such host"*** delete cluster greptime-jzuuke `kbcli cluster delete greptime-jzuuke --auto-approve --namespace ns-oxzrz ` Cluster greptime-jzuuke deleted pod_info:greptime-jzuuke-datanode-0 1/1 Running 0 25s greptime-jzuuke-datanode-1 1/1 Running 0 42s greptime-jzuuke-etcd-0 2/2 Running 0 20m greptime-jzuuke-etcd-1 2/2 Running 0 18m greptime-jzuuke-etcd-2 2/2 Running 0 22m greptime-jzuuke-frontend-0 1/1 Running 0 21m greptime-jzuuke-frontend-1 1/1 Running 0 18m greptime-jzuuke-meta-0 1/1 Running 0 19m No resources found in ns-oxzrz namespace. delete cluster pod done No resources found in ns-oxzrz namespace. check cluster resource non-exist OK: pvc No resources found in ns-oxzrz namespace. delete cluster done No resources found in ns-oxzrz namespace. No resources found in ns-oxzrz namespace. No resources found in ns-oxzrz namespace. Greptimedb Test Suite All Done! --------------------------------------Greptimedb (Topology = Replicas 2) Test Result-------------------------------------- [PASSED]|[Create]|[ClusterDefinition=greptimedb;ClusterVersion=greptimedb-0.3.2;]|[Description=Create a cluster with the specified cluster definition greptimedb and cluster version greptimedb-0.3.2] [PASSED]|[Connect]|[ComponentName=frontend]|[Description=Connect to the cluster] [PASSED]|[VerticalScaling]|[ComponentName=frontend,etcd]|[Description=VerticalScaling the cluster specify component frontend,etcd] [PASSED]|[Restart]|[ComponentName=frontend,etcd]|[Description=Restart the cluster specify component frontend,etcd] [PASSED]|[Stop]|[-]|[Description=Stop the cluster] [PASSED]|[Start]|[-]|[Description=Start the cluster] [PASSED]|[Restart]|[ComponentName=meta]|[Description=Restart the cluster specify component meta] [PASSED]|[Restart]|[-]|[Description=Restart the cluster] [PASSED]|[VerticalScaling]|[ComponentName=datanode]|[Description=VerticalScaling the cluster specify component datanode] [PASSED]|[VerticalScaling]|[ComponentName=meta]|[Description=VerticalScaling the cluster specify component meta] [PASSED]|[VolumeExpansion]|[ComponentName=datanode;ComponentVolume=datanode]|[Description=VolumeExpansion the cluster specify component datanode and volume datanode] [PASSED]|[VolumeExpansion]|[ComponentName=etcd;ComponentVolume=etcd-storage]|[Description=VolumeExpansion the cluster specify component etcd and volume etcd-storage] [PASSED]|[HorizontalScaling Out]|[ComponentName=frontend,datanode]|[Description=HorizontalScaling Out the cluster specify component frontend,datanode] [PASSED]|[HorizontalScaling In]|[ComponentName=frontend,datanode]|[Description=HorizontalScaling In the cluster specify component frontend,datanode] [PASSED]|[Restart]|[ComponentName=datanode]|[Description=Restart the cluster specify component datanode] [PASSED]|[Update]|[TerminationPolicy=WipeOut]|[Description=Update the cluster TerminationPolicy WipeOut] [PASSED]|[Delete]|[-]|[Description=Delete the cluster] [END]