Kubernetes Pod Consolidation Fights Its Own Placement Defaults
A node sitting at 13% processor utilization is common on a cluster that’s never repacked its own workloads, and it still costs exactly as much as a node running at capacity. ZopDay’s new Pod consolidation setting, on a ROSA cluster’s Settings tab under Automatic Scaling, packs pods onto fewer nodes so the autoscaler has empty ones it can actually remove. The setting itself is simple. Getting it to hold against the cluster’s own defaults was not.
Consolidation only matters as a feeder to node removal, since node scheduling is the practice that determines whether a cluster’s capacity matches what it actually runs. A pod moved off a node accomplishes nothing on its own. The autoscaler still has to see that node sit empty long enough to remove it, and that’s exactly the step the cluster’s own defaults were quietly blocking.
Why Consolidation Has to Reach Inside the Cluster Itself
Every other setting on the Automatic Scaling tab manages the ROSA cluster from outside, through OCM, the Red Hat OpenShift Cluster Manager API that ZopDay normally talks to. Consolidation can’t. OCM has no lever for it, because the software that performs consolidation is an optional add-on to ROSA, not a built-in part of the managed service. ZopDay installs it through the cluster’s own operator catalogue the first time the setting is switched on, running the install as a provisioning job with no hardcoded manifests, using the per-cluster connect credential to reach the cluster directly.
That distinction shapes what happens when the setting changes. A cluster holds exactly one consolidation configuration. If that configuration already exists and ZopDay didn’t create it, the screen treats it as read-only rather than overwriting it. Switching consolidation off removes the configuration object but leaves the add-on itself installed, because removing the add-on would delete its entire resource type, and every consolidation object that type ever held, cluster-wide.
The Scheduler’s Own Defaults Were Undoing It
Here’s the finding that mattered most. A ROSA cluster’s default placement behavior puts new pods on whichever nodes have the most free capacity, which is precisely the opposite of what consolidation is trying to do. During live testing, a pod got moved off an under-used node as consolidation intended, and its replacement landed straight back on that same node a minute later. Twice. Call that a placement undertow: the default scheduler behavior pulling a pod back to the exact node consolidation just emptied it from, invisibly undoing the work in the same cycle it happened.
The add-on’s own safety guard didn’t catch the undertow, because that guard only fires when a cluster operator has deliberately set a conflicting placement policy. Nobody had. The conflict came from the cluster’s factory default, which the guard was never built to see. This works when a cluster’s placement profile already favors tight packing. It breaks, silently, when the default favors spreading load instead, since consolidation’s own logic has no way to tell the two apart. ZopDay’s read of a cluster’s consolidation status now also reports its placement profile, and the setup screen warns explicitly when that profile is set to work against consolidation instead of with it.
| What each mechanism optimizes for | Consolidation’s goal | Default placement’s goal |
|---|---|---|
| Where a new pod lands | Fewest nodes holding work | Most free capacity per node |
| Effect on an emptied node | Stays empty, becomes removable | Refills with the next pod scheduled |
| Who has to notice the conflict | The operator reading the warning | Nobody, by default |
The Savings Are Real, Just Smaller Than the Ticket Assumed
Even with placement aligned, node consolidation doesn’t reclaim as much as the feature’s original ticket examples suggested. A node running at 13% processor utilization still gets skipped at the setting’s middle aggressiveness level, because system pods (the platform’s own daemons and controllers) hold memory usage above the line the add-on checks before it will move anything off. Consolidation packs work tighter. It doesn’t touch the floor of memory every node reserves just to run the cluster itself.
The real lever stays the autoscaler’s own node-removal threshold, the setting that decides how empty a node has to sit before it’s actually terminated. Consolidation’s job is to feed that threshold real, empty nodes more often. It was never going to replace it, and the release scoped the claim down to match what live testing actually measured rather than what the ticket had assumed.
A second, smaller finding closed a silent way for the feature to switch itself off: the run interval field, how often consolidation re-evaluates the cluster, could be saved empty. Doing so switched the entire feature off and left the cluster reporting itself unhealthy, while the save action itself reported success. The field can no longer be submitted blank.
Three Bugs the Feature Found in Itself
Three separate defects surfaced only once the feature was driven against a real cluster instead of a mocked one. The add-on could be installed in a state that left it dead on arrival, because the readiness check ZopDay relied on only confirmed that the add-on’s resource type had been published to the cluster’s API, which happens even when the underlying install fails. The check now requires an actual running deployment before it reports success.
Separately, the config lookup for an existing consolidation setup sent the wrong account identifier, so a real configuration on the cluster simply wasn’t found. And on the setup screen, the namespace picker’s checkboxes were wired to store the raw browser event object instead of the value a person had actually selected, silently breaking which namespaces the configuration would apply to.
A Shared Component Failed on Every Screen That Used It
A UI review pass on the feature filed 21 findings, and most of them weren’t about consolidation at all. They were in a searchable picker component this feature reuses, one that also appears on other ZopDay screens. Its dropdown options couldn’t be reached by keyboard navigation at all. A user viewing the screen in read-only mode could still clear a selection and trigger an unsaved-changes prompt, because the picker’s clear control was a clickable span nested inside a button, a structure a disabled form fieldset doesn’t actually cover. The picker’s trigger element had no accessible name for a screen reader to announce, and its keyboard focus outline measured 2.02:1 contrast against the background in light theme, below what’s legible, now corrected to 4.55:1.
None of those defects were specific to consolidation. They were sitting in a component several other features already depended on, waiting for whichever feature got a UI review first. Fixing the picker once fixed all of them, on every screen that uses it, not only the one where the review happened to start.
What Was Verified Live, and What Wasn’t
The fix touched the ZopDay frontend, the cluster-scaling backend path, and the shared picker component itself, and it was verified by driving the actual UI against a real ROSA cluster: install, configure, reconfigure, remove, a genuine pod eviction, and the take-over screen for an existing configuration. Two scenarios weren’t run against a live cluster. The read-only screen for a user without edit permissions was checked with mocked data only, since exercising it for real needs a second account with restricted access. And the refusal path, what happens when ZopDay tries to take over a configuration a cluster already has, wasn’t tested against a cluster that actually had one waiting.
Naming those two gaps in the release notes is itself the point. A feature that reaches inside a cluster’s own scheduler has more ways to fail quietly than one that only reads status back through an API, and the honest account of what wasn’t checked is worth more than a claim that everything was.
