Kubernetes Cost Scheduling Strategies by Cluster Topology

Kubernetes cost scheduling means removing compute from a cluster during hours when its workloads are not needed and restoring it before they are. Unlike a single VM, a cluster has several layers you could act on: the control plane, node pools, namespaces and individual workloads. The right layer follows from two facts about your cluster: how it is billed, and who shares it.

Start with the bill

Three pricing models cover the major managed offerings:

• Node-billed. On GKE Standard node pools you pay for the underlying Compute Engine instances until the nodes are deleted; on EKS you pay for the worker-node resources such as EC2 instances, EBS volumes and public IPv4 addresses, and on AKS you pay for the node VMs plus, on the Standard tier, $0.10 per cluster per hour (the Free tier has no management fee). Removing Pods saves nothing until the nodes go.
• Pod-billed. GKE Autopilot general-purpose workloads are charged per second for the CPU, memory and ephemeral storage that running Pods request, with no minimum duration. AWS Fargate charges for vCPU, memory and storage from the time the image download starts until the Pod terminates, rounded up to the nearest second. Removing Pods saves immediately.
• Per-cluster fee. EKS charges US$0.10 per cluster per hour for standard Kubernetes version support, and GKE charges a flat US$0.10 per cluster per hour management fee (with a $74.40 monthly free-tier credit covering one zonal or Autopilot cluster per billing account), and the AKS Standard tier charges $0.10 per cluster per hour. None of these fees is affected by scaling nodes to zero.

Strategy 1: dedicated cluster per environment

If dev and staging each have their own cluster, schedule each cluster as a whole: scale its node pools to zero outside working hours and back up before the team starts. This is the simplest model and removes the node cost in one action. The per-cluster fee remains on EKS and GKE, so many small dedicated clusters carry a fixed overhead each.

Strategy 2: shared cluster, node pool per environment

On a shared cluster, give each schedulable environment its own node pool and pin its workloads to it with node selectors or affinity. Then schedule the node pool: the qa pool scales to zero at night while the platform pool keeps ingress controllers and operators running. Pair this with scaling the environment’s workloads down first, so Pods are not left pending on a pool that no longer exists.

Strategy 3: shared cluster, namespace or workload schedules

When environments share nodes, schedule at the namespace level and let the cluster autoscaler remove the nodes that empty out. See namespace-level scheduling for the object-by-object mechanics and namespace vs cluster scheduling for the trade-off. Finer still, individual workloads can be scheduled: ZopNight treats EKS and GKE Deployments and StatefulSets (scaled by replicas) and CronJobs (suspended and resumed) as schedulable resource types in their own right.

Strategy 4: Pod-billed platforms

On Fargate or Autopilot there are no nodes to manage, so the schedule acts on workloads: scale Deployments and StatefulSets to zero and suspend CronJobs. Because billing follows the Pods, this is the most direct saving of the four.

The persistent volume traps

Stateful workloads are where Kubernetes schedules go wrong.

• Zone affinity. An Amazon EBS volume can only be attached to an instance in the same Availability Zone. If a node pool scales back up with nodes in a different zone, a Pod bound to that volume cannot run there.
• Binding mode. With the default Immediate binding, volumes can be provisioned without knowledge of the Pod’s scheduling constraints, which can leave Pods unschedulable. volumeBindingMode: WaitForFirstConsumer delays binding and provisioning until a Pod using the claim is created, so the volume follows the Pod’s topology.
• Reclaim policy. If a StorageClass does not set reclaimPolicy, volumes it provisions default to Delete. With Retain, a PersistentVolume survives deletion of its claim and must be reclaimed manually, a safer setting for data you cannot recreate.

Scaling a StatefulSet to zero keeps its PersistentVolumeClaims, so the data waits for the replicas to return; the traps above only bite when nodes come back somewhere the volume cannot follow.

For how ZopNight schedules node groups specifically, see the EKS node group and GKE node pool schedule pages.