The subscribable event catalogue

Outcome

By the end of this lesson, you will be able to navigate the eight notification event categories, explain why platform alerts are one key rather than many, and recognise the two one-time lifecycle emails and why an established org never receives them.


Tier	Operator
JTBD	”Subscribe to exactly the events my team needs, and know what exists to subscribe to.”
Personas	Platform Engineer · SRE · FinOps Lead
Prerequisites	M1.6.L3 (notifications)
Time	9 minutes
Bloom verb	Navigate (Apply), Explain (Understand), Recognise (Remember)

1. Concept

You cannot invent an event. There is a fixed list of what ZopNight will tell you about, and the Alerts page is that list laid out as a grid.

It runs to eight categories:

SCHEDULING
  resource.started            (scheduled)
  resource.started.manual
  resource.stopped            (scheduled)
  resource.stopped.manual
  resource.group.started.manual /.stopped.manual
  resource.failed
  resource.retry_exhausted
  resource.override           (Set) /.cleared /.expired

AUTOMATION
  autoscaler.policy_lifecycle
  eventreadiness.lifecycle

BUDGETS & BILLING
  budget.warning /.exceeded
  budget.ai_gateway.warning /.critical
  billing.sync_degraded

COST ANOMALY
  cost.anomaly.{warning,critical,emergency}
  at org / cloud-account / resource-group / resource /
  team scope

REMEDIATION
  remediation.approval_required /.completed /.failed
  recommendation.ticket_opened

REPORTS & EXPORTS
  export.ready

CLOUD ACCOUNTS
  permission.denied

PLATFORM ALERTS
  resource.alert
  platform.alert.provider_incident

Start and stop are split by trigger

resource.started         a SCHEDULE did it
resource.started.manual  a HUMAN did it

They are separate keys so you can subscribe to each independently, and most teams should. A schedule firing on time is routine; someone manually starting a production resource at 2 AM is not, and mixing them into one subscription buries the second in the first.

Overrides are owned by Config rather than the executor, which is what stops resource.override being notified twice for one action.

Platform alerts are deliberately one key

This is the subtlest thing in the catalogue and it is worth understanding, because the design is counter-intuitive.

resource.alert covers EVERY category and ALL SIX
producing services.

The per-category keys that used to exist were retired.

The reason is a race:

The cross-service notify gate COLLIDES ON THE FINGERPRINT,
so exactly ONE service dispatches a given root cause.

WHICH service wins that race is not deterministic.

So under per-category keys, WHICH KEY paged the customer
depended on who won:
  a "Cloud Alert" subscriber went silent whenever the
  aggregator won
  a per-category subscriber went silent whenever anyone
  else did

One root cause detected by several services has to be one subscription for the customer, or the subscription is unreliable in a way nobody can debug. No migration was needed to retire the old keys because they never shipped, so no stored subscription can reference one.

platform.alert.provider_incident KEEPS ITS OWN KEY,
precisely because it has NO RACE: fleet-wide correlation
runs only in the aggregator.

A distinct key is what lets the customer be told
"this is the provider, not you".

That last sentence is the whole justification. During a provider incident, the most valuable thing a notification can do is stop your team debugging their own infrastructure.

permission.denied, and its careful firing rule

Emitted by the discoverer, not the aggregator. Each per-permission probe cycle diffs the freshly-probed rows against the previously-persisted ones.

FIRES when a permission NEWLY transitions into denied.
  A "newly denied" means a permission|region key not seen
  in the previous set.

CARRIES the WHOLE CURRENT DENIED SET, not just the delta,
grouped into feature and resource-type groups with the
last-probed time.

DOES NOT re-send when the denied set is unchanged since
the last cycle.

Two details make the card trustworthy:

1. It is computed from the ZopNight catalog ONLY, never
   the ZopDay one, and it skips permissions the UI hides.
   So the card's Denied Count MATCHES what you see in the
   app. A card that disagreed with the UI would be worse
   than no card.

2. It resolves the friendly cloud-account name, falling
   back to the account id.

The “whole set, not the delta” choice is the right one for an operator: when a new denial appears you want the current picture, not a fragment you have to reassemble from three previous emails.

export.ready

Fires when a cost-report, audit-log or recommendation
export finishes. The card carries the SIGNED DOWNLOAD
LINK.

This is what makes a long export usable: you start it, close the tab, and the notification brings you the link. Worth subscribing to if anyone in your org runs year-long audit exports.

The two lifecycle emails

These are not alerts, and the distinction is structural:

ONBOARDING COMPLETE
  when an org's first cloud scan finishes
  reports: accounts connected, resources discovered,
           services monitored

RECOMMENDATIONS READY
  when the first savings recommendations are priced and
  have settled
  reports: estimated monthly savings, a per-category
           breakdown, the highest-impact items

THEY GO THROUGH THE NOTIFICATION SERVICE BUT NOT THROUGH
THE ALERTING POLICY.

Nothing failed, so they carry NO SEVERITY and NEVER
ESCALATE.

Each sends ONCE PER ORG, EVER, claimed durably so a retry
cannot double-send.

Recipients default to org admins and can be switched to all members per org, and the existing per-org email opt-out applies.

ONLY ORGS CREATED AFTER THE FEATURE SHIPPED RECEIVE THEM.

A backfill migration muted every org that already
existed, so no established customer is emailed about
setup they completed months ago.

If you are an established customer wondering why you never saw these: that is why, and it was deliberate.

2. Demo

Designing a subscription matrix for a platform team:

TEAM: 9 engineers, 1 on-call rotation, 3 clouds

#platform-noise          (nobody is paged from here)
  resource.started            scheduled starts
  resource.stopped            scheduled stops
  export.ready                someone's export finished

#platform-ops            (working hours attention)
  resource.started.manual     WHY did a human start this
  resource.stopped.manual
  resource.failed
  autoscaler.policy_lifecycle
  permission.denied           a NEW denial appeared
  billing.sync_degraded

#platform-alerts         (on-call sees these)
  resource.retry_exhausted    retries gave up
  remediation.approval_required
  cost.anomaly.critical       at org + cloud-account scope
  budget.exceeded

PAGERDUTY                (wakes someone up)
  cost.anomaly.emergency
  platform.alert.provider_incident
  resource.alert

THE TWO DECISIONS THAT MATTERED

1. SPLITTING.manual FROM scheduled
   The scheduled keys produce hundreds of events a day
   and are pure noise. The manual ones produce two or
   three a week and every one is worth a glance. Under
   one subscription the manual events were invisible.

2. platform.alert.provider_incident STRAIGHT TO PAGERDUTY
   During an AWS event three months earlier, the team
   spent 40 minutes debugging their own infrastructure.
   This key exists to say "this is the provider, not
   you", and it has no race behind it, so it is
   reliable enough to page on.

WHAT THEY GOT WRONG FIRST
   They subscribed #platform-ops to resource.override,
   expecting a handful. Overrides during an incident
   week produced 60 events. Moved to #platform-noise and
   they review it weekly instead.

3. Hands-on (6 min)

1. Open the Alerts matrix. Count the event types
   available:                                 ______
   Which categories do you currently subscribe to?
   ______________________________________________

2. Are you subscribed to resource.started and
   resource.started.manual through the SAME channel?
                                              Y / N
   If Y, that is the first thing to split.

3. Are you subscribed to permission.denied?    Y / N
   If N, subscribe now. It is the signal that catches a
   silently-narrowed credential, and it only fires on a
   NEW denial rather than repeating.

4. Are you subscribed to
   platform.alert.provider_incident?           Y / N
   Where does it route?  ______________________
   (This is the one that stops your team debugging a
   provider's outage.)

5. Check your org's email preferences. Are lifecycle
   emails going to admins or all members?
   ______________________________________________
   If your org predates the feature, you will not have
   received them at all.

4. Knowledge check

Q1

Why is there a single resource.alert key rather than a per-category platform alert key?

A. To reduce the number of subscriptions to manage
B. Because the categories were found to overlap
C. Because per-category keys could not carry a severity level, so routing by severity would have been quite impossible to build on top of that key scheme itself
D. Because the cross-service notify gate collides on the fingerprint, so exactly one of six services dispatches a given root cause, and which one wins is not deterministic

Show answer

Correct: D. Under per-category keys, which key paged the customer depended on who won that race, so a subscriber went silent whenever a different service won. platform.alert.provider_incident keeps its own key precisely because it has no race: fleet-wide correlation runs only in the aggregator. A distinct key is what lets a customer be told “this is the provider, not you”.

Q2

A permission.denied card fires. What does it contain?

A. The whole current denied set, grouped into feature and resource-type groups with last-probed times, even though the card only fires when a new denial appears
B. Only the permission that was just denied
C. A link to the permissions drawer, with no detail
D. Every permission probed during that cycle, whether it was denied or not, so that the reader can see the whole permission picture all in one go

Show answer

Correct: A. It is computed from the ZopNight catalog only and skips permissions the UI hides, so its Denied Count matches what you see in the app. Sending the whole set rather than the delta is the right call for an operator: when a new denial appears you want the current picture rather than a fragment to reassemble from earlier emails. An unchanged denied set is not re-sent at all.

Q3

An established customer has never received the “Onboarding Complete” or “Recommendations Ready” emails. The reason:

A. They opted out of marketing email
B. They were sent but filtered as automated mail
C. A backfill migration muted every org that already existed when the feature shipped, so only orgs created afterwards receive them
D. The emails only send to those organisations that have more than one cloud account already connected to them by now

Show answer

Correct: C. Each sends once per org ever, and both go through the notification service rather than the alerting policy, carrying no severity and never escalating. Emailing an established customer about setup they completed months ago would be worse than not sending at all, which is why the mute was part of shipping rather than a follow-up.

5. Apply

Split .manual from scheduled start and stop events today if you have not: it is the single highest-value change to most subscription matrices, because manual actions on production are exactly what you want visible and they are currently buried.

Subscribe permission.denied somewhere a human reads within a day. A silently narrowed credential degrades discovery, recommendations and verification at once, and this is the signal that names it.

Glossary terms touched

Event catalogue · export.ready · permission.denied · Lifecycle email

Outcome

1. Concept

Start and stop are split by trigger

Platform alerts are deliberately one key

permission.denied, and its careful firing rule

export.ready

The two lifecycle emails

2. Demo

3. Hands-on (6 min)

4. Knowledge check

Q1

Q2

Q3

5. Apply

Related lessons

Glossary terms touched

Start with the bill.