# Rollbacks

> One-click rollback semantics; build skipped, secrets stay in your cloud, audit trail captures every reversal.

Source: https://zop.dev/docs/zopday/rollbacks

---

## When to roll back

Roll back when a deploy made it past the health gates but is now misbehaving in production; elevated error rate, latency regression, a missed dependency. ZopDay's rollback is **build-skipped, secrets-preserved, audit-trailed.**

For deploys that fail their own health gates, **ZopDay rolls back automatically**: no human action required. Manual rollback covers the case where the regression appears after the deploy is marked live.

## How rollback works

Every successful release is kept on the cluster for the configured grace window. Rolling back is a deterministic re-point:

1. **Select the target release.** The pipeline view shows the last N successful releases with their commit SHA, deployer, and timestamp. Pick the one you want.
2. **Pre-flight check.** ZopDay verifies the target release still exists on the cluster and that its supporting resources (ConfigMaps, Secrets references) are intact.
3. **Apply.** The selector flips back to the target release. For rolling deploys, replicas swap progressively. For blue/green, the swap is atomic.
4. **Gate.** Health gates run against the rolled-back version; readiness, probes, smoke tests.
5. **Audit.** A `pipeline.rollback` event captures actor, source release, target release, timestamp, and reason.

**No rebuild happens.** The image hash is the one that was already on the cluster; you are reverting, not redeploying.

## Strategy-specific behaviour

### Rolling

Rollback replays the rolling-update logic in reverse; the previous ReplicaSet is scaled up while the current one is scaled down, batched by the same `maxSurge` / `maxUnavailable` values. Typical completion time matches the original rolling deploy.

### Canary

If rollback happens **before** the canary reaches 100%, it's an immediate stop; the canary ReplicaSet is scaled to zero and traffic shifts fully to the stable version. Sub-second flip.

If rollback happens **after** the canary reached 100% (i.e. it became the new stable), the previous stable release is restored as a fresh rolling update.

### Blue / Green

The selector flips back to the previous (blue) release, the last known-good version. This is **atomic**: every in-flight request after the flip lands on the rolled-back version. Old replicas are kept warm until the next deploy or the configured grace window elapses, whichever comes first.

## What survives a rollback

- **Secrets in your cloud vault** are untouched. The rolled-back version reads the same vault path as before.
- **ConfigMaps** managed by the previous Helm release are re-applied.
- **Persistent volumes** are not touched. Database migrations are **not reversed**: you need to run a forward-compatible migration strategy if you want rollback to be fully safe (additive columns, expand-then-contract).
- **In-flight HTTP requests** complete on their original version (grace shutdown).

## Safety gates

Before any rollback applies, the following gates run:

1. **Authorization**: the actor must have `Editor` or higher on the target environment (rollback is a write action).
2. **Quiet window**: if a release window has been declared as quiet (release freeze), rollback requires an explicit `--override` flag.
3. **Target still exists**: the rolled-back version's image must still be pullable. Expired images fail the gate.
4. **Dependency check**: if the rolled-back version depends on a config / secret / CRD that has since changed, the gate surfaces the divergence and asks for confirmation.

If a gate fails, the rollback is **blocked, not retried**: a human decision is required.

## Audit

Every rollback ships a structured audit entry:

```json
{
  "event": "pipeline.rollback",
  "actor": { "type": "user", "email": "ops@example.com" },
  "service": "payments-api",
  "environment": "prod",
  "sourceRelease": "v2.41.0",
  "targetRelease": "v2.40.3",
  "reason": "elevated 5xx in monitor after v2.41",
  "appliedAt": "2026-06-01T05:01:23Z",
  "gates": {
    "authorization": "pass",
    "quietWindow": "pass",
    "targetExists": "pass",
    "dependencyCheck": "pass"
  }
}
```

The same record is forwardable to your SIEM on the Enterprise tier.

## API and CLI

```bash
# CLI
zopday rollback payments-api --to v2.40.3 --reason "elevated 5xx in monitor"

# During a declared quiet window (release freeze), add --override to proceed
zopday rollback payments-api --to v2.40.3 --reason "elevated 5xx in monitor" --override

# API
curl -X POST https://api.zop.dev/v1/pipelines/payments-api/rollback \
  -H "Authorization: Bearer $ZOPDAY_PAT" \
  -d '{"target":"v2.40.3","reason":"elevated 5xx in monitor"}'
```

Generate `$ZOPDAY_PAT` as a Personal Access Token from your profile menu; see [Authentication](https://zop.dev/docs/zopday/authentication#personal-access-tokens).

Both surface the same gates and emit the same audit record.