Outcome
By the end of this lesson, you will be able to explain why an accepted Smart Tag can disappear without anyone revoking it, diagnose the three causes, and decide whether a given drift event is the tag’s problem or the estate’s.
| Tier | Engineer |
| JTBD | ”Understand why last month’s attribution changed when nobody touched the policy.” |
| Personas | Platform Engineer · FinOps Lead |
| Prerequisites | M2.8.L3 |
| Time | 9 minutes |
| Bloom verb | Explain (Understand), Diagnose (Analyze), Decide (Evaluate) |
1. Concept
The rule that governs everything in this lesson is one sentence:
An accepted tag is kept only while it is still derived, with the same value.
Reconciliation runs on every refresh. It is not a periodic cleanup job you can schedule around; it is a condition that is continuously true.
ON EVERY REFRESH, FOR EVERY ACCEPTED TAG: is the policy still deriving this key on this resource? no -> the accepted tag is dropped is it still deriving THE SAME VALUE? no -> the accepted tag is dropped; the new value appears as pending yes to both -> the accepted tag survivesThis is the opposite of how a cloud tag behaves. A cloud tag persists until someone changes it. A Smart Tag persists only while its derivation still holds. That difference is the source of every surprise in this lesson.
The three ways an accepted tag disappears
1. THE RESOURCE CHANGED The policy derives from provider, region, type, instance_type or name. Change one of those and the derived value changes with it.
Renamed payments-api-03 -> checkout-api-03 team = payments (accepted) is dropped team = checkout appears as pending Nobody did anything wrong. The derivation followed the rename, which is what it is for.
2. THE POLICY CHANGED Editing the policy re-derives everything it touches. A change to the derivation rule invalidates the accepted values that came from the old rule.
Widening a name policy from the leading segment to the first two segments changes EVERY value it produces. Every accepted tag under that key drops at once.
3. THE CLOUD TAG CHANGED This one only affects the auto-accept path. A tag that was auto-accepted because it matched the live cloud tag stops matching when someone edits the cloud tag, and returns to pending.The one that looks like a bug and is not
Cause 2 produces the alarming version: an admin makes a small policy edit and thousands of accepted tags vanish, taking the attribution with them and pushing spend back into Unattributed.
WHAT IT LOOKS LIKE Monday: Unattributed 9% Tuesday: Unattributed 34%, nobody revoked anything
WHAT HAPPENED Someone edited the team policy's derivation rule. Every value it had produced was invalidated, so every accepted tag under that key dropped and reappeared as pending under a new value.
WHY IT IS CORRECT The alternative is worse: keeping the old accepted values would mean your reports attribute spend using a rule that no longer exists, and nothing in the UI would tell you.
WHAT TO DO Re-accept by value group (four or five clicks, per L3), not by resource. Then treat policy edits as a change with a blast radius, and make them deliberately.Treat a policy edit the way you would treat a schema migration: it is a fleet-wide operation, not a settings tweak.
Whose problem is it
The useful diagnostic question is not “how do I stop the tag drifting.” It is “is the derivation wrong, or is the estate telling me something.”
DRIFT SIGNAL USUALLY MEANS─────────────────────────────────────────────────────────────One resource renamed, tag Normal. Re-accept, or acceptfollowed it the new value group.
A whole group's value A team reorganised, or achanged at once naming convention shifted. The tag is reporting a real organisational change.
Values churn every refresh The policy axis is unstable. Usually a name policy on an estate with generated names. Fix the POLICY (L2), not the queue.
A cloud tag was edited away Someone changed the tag in thefrom the derived value console. Worth asking why before assuming the policy is right and the human is wrong.Only the third row is a defect, and the fix is in the policy.
Reconciliation is not drift detection
Worth separating, because the vocabulary collides with a different feature.
SMART TAGS RECONCILIATION Keeps derived state honest against current resource fields. Runs on every refresh. Has no notion of an "expected" tag: whatever the policy derives now IS the expectation.
IaC / TAG-POLICY DRIFT (T5.M5.1.L4) Compares cloud tags against a declared expectation. Reports a violation when they diverge, and someone is expected to act.Smart Tags never reports a violation, because it has nothing to be violated against. If you need “this resource should have tag X and does not”, that is a tagging-policy violation in the governance sense, not a Smart Tags concern.
2. Demo
A month of reconciliation on a real estate, read as signal rather than noise:
MONTH IN REVIEW: team policy, derived from name (leading segment)
WEEK 1 4 accepted tags dropped 4 resources renamed during a service rename (payments-* -> checkout-*) VERDICT: normal. Accepted the new checkout group.
WEEK 2 0 changes.
WEEK 3 1,781 accepted tags dropped in one refresh CAUSE: a platform admin widened the policy from the leading segment to the first two segments, trying to separate payments-api from payments-worker RESULT: every value changed shape at once (payments -> payments-api, payments -> payments-worker) Unattributed 9% -> 41% for about an hour VERDICT: correct behaviour, poorly sequenced change. Re-accepted by value group: 19 groups, 19 clicks. The new split was actually what they wanted; the surprise was the blast radius, not the outcome.
WEEK 4 41 accepted tags dropped, no policy change, no renames CAUSE: an eksctl-managed nodegroup recycles its nodes weekly with generated names VERDICT: the policy axis is wrong for that set. These will churn forever. Left them pending permanently and attributed that nodegroup by resource group instead.
WHAT THE MONTH TAUGHT Weeks 1 and 3 were the system working. Week 4 was the only real finding, and it was about the estate, not the tags.3. Hands-on (6 min)
1. Note today's accepted count: ______ Note it again after the next refresh: ______ A difference with no policy edit and no renames is worth investigating.
2. Pick a resource with an accepted derived tag. What would have to change for that tag to drop? ______________________________________________ (If you cannot answer, re-read the reconciliation rule; it is the whole lesson.)
3. Before your next policy edit, write down: key being edited: ________________ accepted tags under that key: ________________ That number is your blast radius. All of them will drop on the next refresh.
4. Classify one recent drop: [ ] resource changed [ ] policy changed [ ] cloud tag changed Which of the four DRIFT SIGNAL rows does it match? ______________________________________________ Is the fix in the queue, or in the policy? ______________________________________________
5. Name one tag key in your estate that should NOT be a Smart Tag, because it needs to persist independently of any derivation: __________________________4. Knowledge check
Q1
An admin edits a tagging policy’s derivation rule. On the next refresh, several thousand accepted tags disappear and Unattributed jumps. This is:
A. A bug; accepted tags should be immutable
B. A reconciliation job that failed halfway
C. Correct behaviour
D. A permissions failure
Show answer
Correct: C. An accepted tag is kept only while it is still derived with the same value, so changing the derivation rule invalidates every value it produced. Keeping the old values would mean reports attributing spend by a rule that no longer exists, with nothing in the UI to say so. The practical lesson is to treat a policy edit as a fleet-wide change with a blast radius equal to the accepted count under that key, and to re-accept by value group rather than by resource.
Q2
A Smart Tag key churns on almost every refresh: values change, accepted tags drop, new pending values appear, with no policy edits. The right fix:
A. Re-accept the new values each week
B. Change the policy
C. Disable reconciliation for that key
D. Apply the tags to the cloud so they persist
Show answer
Correct: B. Constant churn means the derivation axis is unstable for that resource set, typically a name-derived policy over cloud-generated or recycled names. Working the queue harder treats the symptom; the axis is the defect. D is a tempting trap: applying to the cloud does make the cloud tag persist, but the derived value keeps churning underneath and the two now disagree. C is not available, and would not be desirable if it were.
Q3
How does Smart Tags reconciliation differ from the tag-drift detection covered in T5.M5.1?
A. Reconciliation keeps derived state honest against the resource’s current fields and has no notion of an expected tag: whatever the policy derives now is the expectation, so it never reports a violation
B. They are the same feature under two names
C. Reconciliation runs weekly while drift detection runs daily, so a tag that changes in between the two is only ever noticed on the slower of the two cycles on each pass through it
D. Reconciliation only applies to AWS
Show answer
Correct: A. Drift detection compares cloud tags against a declared expectation and does report violations for someone to act on. The distinction matters when someone asks Smart Tags to tell them which resources are missing a required tag. It structurally cannot: it has nothing to be violated against. That question belongs to tagging-policy governance.
5. Apply
Before your next policy edit, check the accepted count for that key on Smart Tags. That number is what will drop on the next refresh, and knowing it in advance turns a surprise into a planned change.
Set a monthly habit of reading the drop events as signal: renames and reorganisations are the estate telling you something, and only constant churn is a defect in the policy.
Related lessons
- L3: Accept, revoke, apply to cloud
- L1: What Smart Tags derive
- T5.M5.1.L4: Drift detection and remediation
- T5.M5.1.L5: Reorg-proof tagging
Glossary terms touched
Reconciliation · Tag drift · Derived tag · Blast radius
Complete M2.8 → 10-question module quiz.