# Provisioning overview

> How ZopDay provisions clusters, datastores, networking, Helm components, and VMs; one state machine, idempotent steps.

Source: https://zop.dev/docs/zopday/provisioning/overview

---

ZopDay's provisioner runs every infrastructure creation through one state machine: `provisioning_jobs`. Five resource families, three clouds, one ledger. This page covers the shape of the system; the per-family details are on the next pages.

![Provision Infrastructure wizard with a 5-step flow (Resource Type, Account & Region, Network, Configuration, Review) and the resource picker showing Kubernetes Cluster, MySQL Database, PostgreSQL Database, and Redis Cache options](https://storage.googleapis.com/zopdev-blog-resources/1/files/originals/20260602/4bb7241c-f546-4dfe-956e-08350a09a3a9-zopdayprovisionnew.png)

*Provision Infrastructure wizard; step 1 picks the resource type (cluster, datastore, cache).*

## What the provisioner creates

| Family | What it creates | Providers |
|---|---|---|
| **Clusters** | EKS / GKE / AKS control plane + workers + cluster IAM + default platform Helm components | AWS · GCP · Azure |
| **Datastores** | RDS / Cloud SQL / Azure DB (Postgres + MySQL); ElastiCache / Memorystore / Azure Cache for Redis | AWS · GCP · Azure |
| **Networking** | VPC, subnets, NAT, VPC peering (AWS + GCP), GCP Private Service Connect for cross-VPC Cloud SQL, DNS zones (Route 53 / Cloud DNS / Azure DNS) | AWS · GCP · Azure |
| **In-cluster components** | Helm catalog (`configs/components.yaml`) installed via Helm SDK on a registered space | Any registered space |
| **VMs** | Production-shaped multi-service VM environment with NAT outbound, optional cloud LB, no SSH (IAM-gated exec only) | AWS · GCP (Azure VMSS in preview / not yet GA) |

Each family has provider parity on day one (with the noted exceptions). The same job state machine drives all of them.

## Two job kinds, one table

`provisioning_jobs` carries two job kinds:

| `kind` | Used for |
|---|---|
| `provisioning` | Infra creation + BYO datastore adoption |
| `remediation` | Auto-remediation workflows (ZopNight; see [Auto-remediation](https://zop.dev/docs/zopnight/optimization/auto-remediation)) |

Mixing them in one table lets the state machine, reconciler, and audit log work uniformly. The job's `subject_type` / `subject_id` discriminates what's being acted on.

## Job lifecycle

```
draft → queued → running → completed
                  ↓
                  failed → retry (resumes from last successful step)
                  ↓
                  cancelled → terminal
```

Each step inside the job is idempotent. If a pod restarts mid-step, the reconciler picks up where it left off; the step is re-run, and because steps are idempotent, the cloud API call either fails fast on "already exists" or is a no-op.

The reconciler holds a row-level lock with a TTL that exceeds the worst-case step duration. Lock TTL is per-family; cluster provisioning gets a longer TTL than a Helm install because EKS create takes longer than a `helm install`.

## State carried on `InfraConfig`

The provisioner uses the job's `config` JSON (under `InfraConfig`) to carry resource-family-specific state across steps. For VMs that means `VMs[]`, `VMSecurityGroupID`, `LoadBalancerID`, `ScalingGroupID`, `LaunchTemplateID`, `ExistingResourceGroup`, `BootstrapNginxReady`, etc.

No new tables per family. Adding a sixth resource type would extend `InfraConfig` with new fields, not a new schema migration.

## Modes

The provisioner supports three modes per job:

| Mode | What it does |
|---|---|
| **Create** (default) | Provision new infrastructure end-to-end. Standard path. |
| **Adopt** | Ingest pre-existing customer-managed infrastructure into ZopDay (resource record + connection wiring) without any create-side cloud calls. Currently supported for datastores. Pairs with a credential-update flow that re-stores the secret reference via Config's `StoreDatastoreCredentials` RPC. |
| **Retry** | Resume a failed job from the last successful step. Re-runs whichever step failed. |

For Adopt jobs, **teardown is a no-op**: ZopDay never owns the lifecycle of adopted resources. Disconnecting an adopted resource only removes ZopDay's pointer; the underlying datastore stays running.

## Pipeline steps

The pipeline is family-specific, but the structure is consistent. For clusters:

```
stepValidate → stepCreateNetwork → stepCreateIAM → stepCreateCluster
            → stepBootstrapComponents → stepRegisterSpace → stepFinalize
```

For VMs:

```
stepValidate → stepCreateNetwork → stepCreateIAM → stepCreateLaunchTemplate
            → stepCreateVMs → [LB sub-steps when LB attached]
            → stepRegisterSpace → stepFinalize
```

The total comes to **7 steps without a load balancer** or **11 steps with a load balancer** attached.

Steps shared across families (`stepCreateNetwork`, `stepCreateIAM`, `stepRegisterSpace`) live in one implementation and branch on `j.ResourceType`. Adding a new resource type slots into the same shared steps without forking them.

The cluster and VM pipelines are shown above; the datastore and networking families follow the same shape, with their per-family step lists documented on the [Datastores](https://zop.dev/docs/zopday/provisioning/datastores) and [Networking](https://zop.dev/docs/zopday/provisioning/networking) pages.

## Endpoint surface

CRUD on jobs:

```
POST   /provisioning-jobs              Create
GET    /provisioning-jobs              List with filters
GET    /provisioning-jobs/{id}         Get one
PATCH  /provisioning-jobs/{id}         Partial update (also used for VM op discriminator)
DELETE /provisioning-jobs/{id}         Cancel + cleanup
POST   /provisioning-jobs/{jobID}/retry   Retry from last failed step
```

Job-scoped database CRUD:

```
GET    /provisioning-jobs/{jobID}/databases
POST   /provisioning-jobs/{jobID}/databases
GET    /provisioning-jobs/{jobID}/databases/{dbID}
DELETE /provisioning-jobs/{jobID}/databases/{dbID}
```

Connections (for service connections + database connections):

```
GET    /provisioning-jobs/{jobID}/connections
POST   /provisioning-jobs/{jobID}/connections
DELETE /provisioning-jobs/{jobID}/connections/{connID}

GET    /provisioning-jobs/{jobID}/databases/{dbID}/connections
POST   /provisioning-jobs/{jobID}/databases/{dbID}/connections
DELETE /provisioning-jobs/{jobID}/databases/{dbID}/connections/{connID}
```

Helm component catalog:

```
GET    /catalog/components                  List available Helm components
POST   /provisioning-jobs/install-component  Install one onto a registered space
```

VM operations (post-provisioning, ride the existing PATCH route with `vmOp` discriminator):

```
PATCH /provisioning-jobs/{id}
{
  "vmOp": "attach_lb"     // or "detach_lb"
}
```

No new route surface for VM ops; the typed `vmOp` field on the existing PATCH route handles attach/detach without sprawling the API.

## RBAC

Provisioning-related policy categories specific to ZopDay infrastructure:

| Policy prefix | Covers |
|---|---|
| `space:{view,create,update,delete}` | Deployment spaces |
| `provisioning:{view,create,update,delete}` | Provisioning jobs |
| `service:{view,create,update,delete}` | In-cluster Helm components |
| `deployment:{view,create,update,delete}` | Orchestrated through the deployer |

ZopDay also adds an `audit-log:view` policy for the [audit log](https://zop.dev/docs/zopday/operations/audit-logs), granted separately. System roles (Admin, Editor, Viewer) inherit them. Custom roles need explicit grants.

## Customer DB safety

ZopDay **never mutates customer-managed databases** via the provisioner; even when ZopDay created the datastore originally, tier changes and replica scaling stay with the customer's DBA team. The provisioner's Pauser registry explicitly excludes `rds*`, `aurora*`, `cloudsql*`, `elasticache*`, `azure-sql`, `postgres*`, `mysql*` from the pause allowlist.

If you want ZopDay to surface sizing recommendations on a DB without changing it, attach it to an [event readiness](https://zop.dev/docs/zopnight/optimization/event-readiness) plan as a monitor-only target. ZopDay computes the recommendation, you implement the change via your normal DB workflow.

## Where to look next

- **[Clusters](https://zop.dev/docs/zopday/provisioning/clusters)**: EKS / GKE / AKS create flow with hardening defaults
- **[Datastores](https://zop.dev/docs/zopday/provisioning/datastores)**: RDS / Cloud SQL / Azure DB / Redis create + adopt
- **[Networking](https://zop.dev/docs/zopday/provisioning/networking)**: VPC, subnets, NAT, peering, DNS
- **[VMs](https://zop.dev/docs/zopday/provisioning/vms)**: production-shape VM environment with NAT outbound + IAM-gated exec
- **[Components](https://zop.dev/docs/zopday/provisioning/components)**: the Helm component catalog