Auto-Remediation Explained: From Detection to Fix
A finding is only worth something once the fix runs. The engineering is in making that fix safe to run on someone else's cloud.
Auto-remediation is the practice of executing the corrective action for a detected problem, such as stopping an idle instance, resizing an oversized one, or adding a lifecycle policy to a bucket, through the provider’s API, instead of stopping at an alert or a ticket. It sits at the far end of the chain that starts with discovery and recommendations. The value is closing the gap between “we know” and “it is fixed”. The risk is that an automated action is wrong in a way no human caught.
What makes a remediation safe
A script that deletes every unattached volume is auto-remediation too; it is just unsafe. Four properties separate a responsible workflow from that script:
- Re-check before acting. The finding may be hours old. The workflow confirms the resource is still in the state the recommendation assumed.
- A gate proportional to the damage. Reversible, low-impact actions can run on one click; destructive ones need someone with authority to approve.
- A narrow action. Each remediation maps to one lever (stop, pause, resize, set a lifecycle policy), not an open-ended script.
- Verification after acting. The workflow confirms the resource reached the intended state and reports clearly when it did not.
How ZopNight structures a remediation
In ZopNight, the Remediate button on a recommendation opens a wizard that runs a workflow of three or four steps: a precondition check, an optional approval step that acts as an admin gate for destructive operations, the cloud action itself, and validation. The available actions are start and stop, pause a service, stop compute, delete a resource, resize (optionally with a stop), set a lifecycle policy, and patch a live Kubernetes manifest.
Rules are wired end to end in one of two modes. Auto runs on one click, with an admin-approval gate when the initiator is not an admin. Guided adds a type-to-confirm review. Single-replica Kubernetes deployments, for example, are scaled from one replica to two only in guided mode, so a workload deliberately run as a singleton is never changed automatically.
Some things are excluded outright. The pause template’s allowlist leaves out RDS, Aurora, Cloud SQL, ElastiCache, Azure SQL, and PostgreSQL and MySQL types, so remediation does not mutate customer databases. Rules that cannot name a concrete lever render a numbered “How to Fix” playbook instead of a Remediate button, and schedule-type findings, such as dev and test compute running around the clock, link to schedule creation with the resource prefilled rather than running a one-shot stop.
Running any of this requires the cloud account to be connected with read and write access. A read-only account still receives discovery and recommendations, and start, stop, and remediation are refused for it before any cloud call is made.
When something fails
Failures are sorted into three categories so the operator knows what to do next: user action (a permission or quota problem, with a fix hint and console link), transient (throttling or a 5xx from the provider, with a retry button), and system (an unsupported API, with a diagnostic to send to support). Only transient failures offer a retry. Account IDs and subscription IDs are stripped from stored error text.
Deciding what to automate
- Start with actions that are reversible: stop, scale-down and most pauses can be undone (a Dataproc pause cannot); delete cannot.
- Require approval where the resource holds data or serves traffic you cannot see from metrics alone.
- Look at the blast radius of a change before approving it.
- Keep an audit trail of every run; see audit trails for cloud changes.
For the resource types that can be paused rather than stopped, see the pause step roster.
Questions we get a lot.
If yours isn't here, email us and we'll answer directly.
Can ZopNight remediation delete resources?
Yes. Delete is one of the remediation actions. Destructive operations go through an approval step that acts as an admin gate before the cloud action runs.
Does auto-remediation work on a read-only connection?
No. Read-only accounts get discovery and recommendations only. Remediation, like start and stop, needs a read and write connection, and ZopNight blocks it server-side otherwise.
Will remediation change my databases?
The pause workflow explicitly excludes RDS, Aurora, Cloud SQL, ElastiCache, Azure SQL, and PostgreSQL and MySQL types, so remediation does not mutate customer databases.