Reading the trigger column · ZopDev University

Outcome

By the end of this lesson, you will be able to interpret every trigger value in the state-history column and distinguish schedule, manual, bulk and override events.


Prerequisites	L1
Time	9 minutes

1. Concept

The trigger column answers the only question anybody asks of a timeline: who or what did this?

Every change carries a trigger, and each one narrows further to the exact cause.

The trigger types

TRIGGER TYPE        EXAMPLE VALUE                       MEANING
─────────────────────────────────────────────────────────────────────
SCHEDULE             schedule:business-hours-eu:<cron id>  Cron in this schedule fired
                     (adds :group:<name> for a group)
MANUAL               manual:user@team.com (or manual:api)  Human action via UI or API
BULK                 bulk:user@team.com:<bulk id>          Member of a group start/stop
OVERRIDE             override:force_on                     Override was set
                     override_expired:force_on             Override ran out
                     override_deleted:force_on             Override was removed early

Those are the only trigger values. Each is rich enough to investigate without further lookup. The schedule name and cron id tell you which schedule and which cron. The user email identifies who acted. The bulk id ties a row to the group action it came from.

Two things never appear in this column. Remediation actions are not part of state history: a remediation runs as its own workflow (precondition, optional approval, action, validation) and is tracked on the recommendation and in the audit log. Track 2 covers it. And a change made outside ZopNight (the cloud console, a pipeline, a script) leaves no row at all, because ZopNight did not carry it out.

Reading SCHEDULE triggers

2026-05-20 08:00 ET   started   schedule:business-hours-eu:<cron id>

A cron in schedule “business-hours-eu” fired
Clicking the trigger value opens the schedule’s detail page
The schedule’s grid shows which cron fired (“0 8 * * 1-5”: the morning start)

Schedule triggers are the expected pattern. A team running healthy schedules sees mostly schedule entries.

Reading MANUAL triggers

2026-05-19 14:23 ET   stopped   manual:jane@zopcloud.com

A human stopped the resource via ZopNight UI or API
Email identifies the actor
Click the trigger to see the audit-log entry with the full request

Manual triggers should be rare for scheduled resources. Frequent manual triggers on a scheduled resource suggest:

The schedule’s hours don’t match the team’s actual needs (adjust schedule)
The team doesn’t trust the schedule (figure out why, address)
A specific use case warrants overrides instead (set them up)

Reading OVERRIDE triggers

2026-05-19 17:30 ET   started   override:force_on
2026-05-22 08:00 ET   stopped   override_expired:force_on

Two related entries: when the override was set and when it expired. Both change state. Setting a force-on override starts the resource straight away. When it expires, ZopNight issues the opposite action to put the resource back in the state its schedule calls for, here a stop. Removing an override early does the same, recorded as override_deleted.

Reading BULK triggers

2026-05-19 18:00 ET   stopped   bulk:jane@zopcloud.com:<bulk id>

Jane started or stopped a whole group in one action
Each member of the group gets its own row, all sharing the same bulk id
Filtering the History page by Source = Bulk shows every member the action touched

A bulk row is a manual action at group scale. If one member of a bulk stop shows failed while the rest show stopped, the group action partly succeeded and that member needs a look.

When the column cannot explain a state

If a resource is running but its last row says stopped, no trigger value will explain it, because something outside ZopNight started it. Common causes:

A terraform apply changed state
A CI/CD pipeline started a resource for a test
An engineer in the cloud console
An out-of-process script

The next stop is the cloud provider’s own log (CloudTrail, Cloud Audit Logs, Activity Log) for that window.

The retry pattern

If ZopNight attempts a transition and the cloud provider returns an error, the executor retries it: after 2 minutes, then 5, then 10. Retries do not get rows of their own. A row is written only when the action succeeds, or once when it finally fails:

Transient error, retry succeeds:
2026-05-19 08:02 ET   started   schedule:business-hours-eu:<cron id>

All three retries fail:
2026-05-19 08:17 ET   failed    schedule:business-hours-eu:<cron id>
                                 "InsufficientInstanceCapacity"

If retries are exhausted (3 retries), the resource stays in its previous state, the action is marked retry_exhausted, and a Retry Exhausted notification fires for anyone subscribed to it.

Trigger-source as a filter

The History page is a filtered table. You can narrow it by status (started, stopped, failed), source (Schedule, User, Bulk, Override), resource and date range. There is no summary panel; the filtered rows are the answer.

FILTER: Source = User, last 7 days, all resources

Result: the same 6 staging RDS instances appear again and again,
        started and stopped by hand several times a day.

If the same few resources keep appearing under Source = User, the schedule probably does not match how the team works. That is a schedule edit, not a discipline problem.

2. Demo

A weekly history review:

T+0       Engineer opens the History page, sets the date range to the
          last 7 days and Source = User.

T+30 sec  Scrolls the rows. Most are one-off starts and stops, but one
          pattern repeats.

T+1:30    Sees pattern: engineer manually starting/stopping the same 6
            staging RDS instances multiple times per day.

T+2 min   Engineer reaches out: "Why are you manually managing those RDS?"
            Response: "The schedule's wrong: staging RDS needs to be on
            evenings for our team's overnight test suite."

T+5 min   Engineer adjusts the staging-eu schedule:
            Cron #1: "0 8 * * 1-5"   Start
            Cron #2: "0 20 * * 1-5"  Stop (existing)
            +Cron #3: "0 22 * * 1-5"  Start (new: for evening tests)
            +Cron #4: "0 4 * * 2-6"   Stop (new: after overnight tests)

T+6 min   Schedule fires on the new hours. The following week, the same
          filter shows those 6 instances only under Source = Schedule.

The history surfaced the operational gap. The fix was a schedule edit, not a process change.

3. Hands-on (5 min)

For your estate’s last 7 days:

1. Open the History page and set the date range to the last 7 days.
2. Set Source = User. Note which resources appear more than twice.
3. Set Source = Override. Note any resource overridden every week.
4. Set Status = Failed. Note any resource that fails repeatedly.
5. For each finding, hypothesize the root cause:
   - Repeated manual actions → schedule's pattern doesn't match real need
   - Weekly overrides → some may be schedule edits
   - Repeated failures → permissions or capacity on that resource
6. Pick one resource you expected to see but did not. If its
   current state does not match its last row, check the cloud
   provider's log for a change made outside ZopNight.
7. Document one improvement to make based on what you saw.

4. Knowledge check

Q1

A History row shows the source bulk:jane@zopcloud.com:<bulk id>. What does it mean?

A. A schedule attached to a group fired its cron for every member of that group at once
B. Jane started or stopped a whole group in one action, and this member is part of it
C. Jane created an override that applied to a group of resources at once
D. A remediation workflow acted on several resources in a single batch

Show answer

Correct: B. A bulk row is one member of a group start or stop that a person ran. Every member gets its own row with the same bulk id. A group schedule writes schedule: rows, an override writes override: rows, and remediation is not written to state history.

Q2

With Source = User, the History page shows the same few resources started and stopped by hand all week. The most actionable next step:

A. Investigate which resources are being manually managed and why
B. Discipline the engineers who keep doing it all by hand
C. Add more notifications
D. Disable manual actions

Show answer

Correct: A. Likely: the schedule’s pattern doesn’t match the team’s real need. Adjust the schedule (or recognize a legitimate need for an override pattern). High manual usage is a signal: usually that the schedule isn’t right for the workload. The fix is to adjust the schedule (or accept the manual pattern as legitimate).

Q3

A scheduled start hits a transient capacity error at 08:00, and the executor’s first retry at 08:02 succeeds. What does the History page show for it?

A. Only a failed row, because the first attempt is the one that gets recorded
B. Three rows: the failed attempt, a RETRY #1 row, and then the successful start
C. One started row; the failed first attempt and the retry do not get rows of their own
D. Nothing at all, because an action that needed a retry is never recorded

Show answer

Correct: C. State history records an action once it succeeds. The executor retries at 2, 5 and 10 minutes, and only if all three retries fail does it write a single failed row and mark the action retry_exhausted. A transient error that a retry absorbs needs no action from the team.

5. Apply

Trigger column reading lives in:

Resource detail → History: per-resource view
History page: every change, filterable by status, source, resource and date

For notifications (covered next), the trigger context shapes the notification routing.

Glossary terms touched

Schedule

Outcome

1. Concept

The trigger types

Reading SCHEDULE triggers

Reading MANUAL triggers

Reading OVERRIDE triggers

Reading BULK triggers

When the column cannot explain a state

The retry pattern

Trigger-source as a filter

2. Demo

3. Hands-on (5 min)

4. Knowledge check

Q1

Q2

Q3

5. Apply

Related lessons

Glossary terms touched

Start with the bill.