A Kubernetes node running ZenML server pods came under memory pressure and the kubelet had to evict pods to reclaim node resources. The cluster autoscaler spun up a new node and all pods were moved to it. This led to a short disruption in service.

We have analyzed this situation and prepared a plan to make ZenML server pods more available going forward. This strategy will be worked upon with priority.