Smart defaults: Welford stats explained

THE PROMISE:
  Engineer doesn't have to know what min/max/target to set
  Smart-defaults compute from actual metrics history
  Output: reasonable starting configuration
  Engineer can accept or override

INPUTS: the CPU datapoints the aggregator holds for the resource
        (the stats are labelled as a 14-day period)
OUTPUTS:
  CPU average:    the aggregator's average (not recomputed)
  CPU stddev:     single-pass Welford over the datapoints
  P90 / P95 / P99: NOT true percentiles. Welford tracks only mean and
                   variance, so the percentiles are a normal-distribution
                   APPROXIMATION: P95 ~= avg + 1.64*stddev,
                   P99 ~= avg + 2.33*stddev (commented "normal approximation"
                   in the code). Accurate for smooth loads; loosest exactly
                   where it matters most, on bursty non-normal workloads.

SMART DEFAULT for CPU-driven target tracking:

  Target value:    3-way tier on P95, not "P95 - 5%":
                     P95 > 85%  -> target 70
                     P95 > 60%  -> target 75
                     else       -> target 80
                    (hotter target when the workload already runs hot)

  Min capacity:    derived from cpu.Avg (the average load), NOT P05.
                    P05 is never computed. The floor covers typical demand.

  Max capacity:    max( current + 2,
                        ceil( (current × P99) / target × 1.3 ) )
                    (1.3 headroom for spikes; and never below count + 2)

  Cooldown:        3-way tier on stddev:
                     stddev > 20  -> 180s
                     stddev > 10  -> 300s
                     else         -> 600s
                    (a HIGHER stddev gets a SHORTER cooldown)

RESOURCE: prod-api ASG
HISTORICAL capacity: 6 (fixed; no autoscaling)

HISTORICAL METRICS (30 days, Welford):
  CPU avg:    45%
  CPU stddev: 15%
  CPU P95:    ~70%   (= 45 + 1.64 x 15, normal approximation)
  CPU P99:    ~80%   (= 45 + 2.33 x 15)

SMART-DEFAULT RECOMMENDATION:
  Target:   P95 (70%) is in (60%, 85%]  ->  75%
  Min:      from cpu.Avg (45%): floor(6 x 0.45 / 0.75) = 3
  Max:      max(6 + 2, ceil(6 x 0.80 / 0.75 x 1.3))
            = max(8, ceil(8.32)) = max(8, 9) = 9
  Cooldown: 300s (stddev 15 is in the middle tier, >10 and <=20)

RECOMMENDATION:
  Min: 3, Max: 9, Target: 75%, Cooldown: 300s

Target = 3-way tier on P95 (70 / 75 / 80)
  Reasoning:
    A hotter workload (high P95) gets a lower target so it scales earlier;
    a cool workload gets a higher target so it does not over-provision.
    The tiers are fixed thresholds, not a P95-minus-margin formula.

Min = floored from cpu.Avg
  Reasoning:
    The AVERAGE load (not a low percentile) sets the floor, so the fleet
    always covers typical demand without constant scale-from-zero churn.

Max = max(count + 2, ceil(count x P99 / target x 1.3))
  Reasoning:
    Headroom for the 1% extremes (1.3x), and never fewer than count + 2

Cooldown adjusted for stddev
  Reasoning:
    Bursty workloads (high stddev) get shorter cooldowns so capacity can
    follow fast swings; smooth workloads (low stddev) get longer ones,
    since there is little to react to

CONTEXT THAT SMART DEFAULTS MISS                  ADJUST
─────────────────────────────────────────────────────────────
Known seasonal spike (Black Friday)               Increase max temporarily
Pending traffic event (new product launch)         Increase max ahead of launch
SLA requires 100ms p99 latency                    Lower target (scale earlier)
Workload is stateful with warm-cache restart      Longer cooldown
Workload is stateless and fast to scale            Shorter cooldown
Cost-aggressive tolerance                          Higher target (run hotter)
External integration constraints (e.g., DB pool)   Lower max
Compliance or auditing constraints                  Lower max in regulated tiers

WELFORD'S ALGORITHM:
  Time complexity: O(n) (linear in samples)
  Space complexity: O(1) (constant memory)

PRACTICAL TIMING:
  30 days of 1-min resolution data (43,200 samples)
  Computation: a single pass, fast enough for an interactive form

SCALES TO:
  Hundreds of targets simultaneously
  Real-time UI response when engineer requests

ZopNight UI: Automate → Auto Scaling → + Create Autoscaling Policy

  Target: select your autoscaling group

  ZopNight queries metrics + computes Welford stats automatically
  Displays:

  SUGGESTED VALUES:
    Min:     3     (computed from cpu.Avg)
    Max:     9     (max(count+2, ceil(count × P99 / target × 1.3)))
    Target: 75%    (3-way tier on P95)
    Cooldown: 300s

  BASED ON:
    CPU metrics (14-day period)
    Avg 45%, P95 ~70%, P99 ~80%, stddev 15%

  [Accept]    [Override]    [Cancel]

GOOD COMBINATIONS:
  Smart-default + engineer fine-tune (most common)
  Engineer adjusts 1-2 values; accepts rest
  Result: better than either alone

  Smart-default + accept as-is (works for many cases)
  No tune needed; defaults work

  Engineer-only (rarely needed)
  When metrics history insufficient
  Or: very specific external context

POOR COMBINATIONS:
  Engineer-only ignoring smart defaults
  Re-implementing the same statistical analysis manually

  Smart-default + override without thinking
  Engineer changes values without considering data

WORKLOAD: production order-processor ASG
  Currently fixed at 8 instances (no autoscaling)

METRICS (last 30 days):
  CPU avg:    51%
  CPU P95:    89%
  CPU P99:    97%   ← consistently saturating during peak
  Stddev:     23%   ← high variance (bursty)

SMART DEFAULT RECOMMENDATION:
  Target: 70%       (P95 89% > 85% -> the hot-workload tier picks 70)
  Min:     5        (from cpu.Avg 51%: floor(8 x 0.51 / 0.70) = 5)
  Max:    15        (max(8+2, ceil(8 x 0.97 / 0.70 x 1.3)) = max(10,15) = 15)
  Cooldown: 180s    (stddev 23 > 20: the shortest tier, reacts fast)

OPERATOR REVIEW:
  Target 70% already suits our latency SLA
    Our p99 latency target is 200ms; 70% CPU leaves headroom
    The hot-workload tier did the right thing automatically
  We do raise Min 5 -> 6 for extra safety margin during peak bursts

  Max 15 acceptable (we don't have hard ceiling)

  Cooldown 180s good (lets capacity follow the bursts)

APPLIED CONFIGURATION:
  Min: 6, Max: 15, Target: 70%, Cooldown: 180s

RESULT (after first week):
  Average instances over week: 6.8 (down from 8 fixed)
  P95 CPU: 78% (within scaling threshold)
  P99 latency: 110ms (within 200ms SLA)
  Cost savings: ~15% vs fixed 8 instances
  Engineer experience: smooth; no manual intervention

LESSONS:
  Smart defaults gave reasonable starting point (~80% optimal)
  Engineer tuning added context (SLA-aware target)
  Final config beats both pure defaults + pure engineer

□ STEP 1: Open Automate → Auto Scaling → + Create Autoscaling Policy

□ STEP 2: Pick a target
  Target name: __________
  Type: __________

□ STEP 3: Read smart defaults
  Suggested min: _____
  Suggested max: _____
  Suggested target: ___%
  Suggested cooldown: ___ seconds
  Based on metrics: __________

□ STEP 4: Identify override candidates
  Target too high for SLA? __________
  Max too high for cost ceiling? __________
  Cooldown right for workload? __________

□ STEP 5: Apply (or save)
  Final config: __________
  Monitor for: 1 week