Outcome
By the end of this lesson, you will be able to map ML/Bedrock cost surface to specific optimization levers, identify the major cost drivers, and recognize which Bedrock rule applies to each pattern.
| Tier | Engineer |
| JTBD | ”Understand the ML cost landscape so I can find the right optimization lever for each cost pattern.” |
| Personas | ML Engineer · Platform Engineer · FinOps Lead |
| Prerequisites | M2.1 (rule library) |
| Time | 9 minutes |
| Bloom verb | Map (Analyze), Identify (Apply), Recognize (Apply) |
1. Concept
Machine learning spending comes in three shapes, and each is reduced differently.
Running a model is charged per request. Training a model is charged by the hour. The machines underneath, usually GPUs, are charged for as long as they exist, whether or not anything is using them.
COST DRIVER TYPICAL % OPTIMIZATION──────────────────────────────────────────────────────────────Bedrock model inference 30-60% Model selection, batch processingFoundation model training 20-40% Spot instances, scheduled runsGPU instance hours 15-30% Right-sizing, schedulingVector DB / embedding store 5-15% Standard storage optimizationSageMaker endpoints 5-20% Right-sizing endpoints; serverlessML cost optimization requires understanding which lever applies where.
Bedrock-specific patterns
Bedrock charges per input token + per output token by model. Different models have wildly different costs:
MODEL INPUT $/M TOKENS OUTPUT $/M TOKENS RELATIVE COST─────────────────────────────────────────────────────────────────────────Claude Opus $15.00 $75.00 1.0x (baseline)Claude Sonnet $3.00 $15.00 0.2x (5x cheaper)Claude Haiku $0.80 $4.00 0.05x (20x cheaper)Titan Express $0.30 $0.40 0.01x (100x cheaper)Llama models $1.00 $2.00 0.05xMistral $0.50-$2.00 $1.00-$8.00 variesA workload using Opus when Haiku would suffice can pay 20-100x what it needs to.
Detection signals
ZopNight detects ML cost patterns:
IDLE BEDROCK PROVISIONED THROUGHPUT: Pattern: capacity reserved but unused Detection: Invocations series absent over a 30-day lookback Rule: RC-1601 (PT Idle): full recovered cost Action: migrate to on-demand, delete the PT
PROVISIONED THROUGHPUT ON A DEPRECATED MODEL: Pattern: PT still pinned to an older in-family generation Detection: modelArn in the deprecated set + a cheaper successor Rule: RC-1610 (PT on Older Model Generation) Action: recreate the PT on the successor, migrate, delete
ORPHANED BEDROCK ARTIFACTS: Pattern: custom models, KBs, imported models nothing references Detection: no invocations / no referencing deployment Rule: RC-1604, RC-1605, RC-1606 Action: guided delete
TRAINING JOBS ON ON-DEMAND: Pattern: training using premium pricing Detection: training workload type + on-demand tier Rule: RC-1621 (SageMaker managed spot training) Action: switch to managed Spot
HIGH INFERENCE COST PER TOKEN: Pattern: high-cost model used for simple queries Detection: NOT DETECTED BY ANY RULE: the Bedrock family reads the control plane, not your prompts or token mix. This is a real lever (L3) with no rule behind it. Action: route by complexity, in your applicationThe 16 Bedrock rules span RC-1601..1610 plus RC-1615 and RC-1630..1634. They sit inside a broader ML family of about 33 rules; the other 17 are SageMaker, RC-1611..1614 and RC-1616..1628. See L2 for the authoritative per-rule list.
Cost drivers per pattern
INFERENCE COSTS (Bedrock or self-hosted): Drivers: model choice, request volume, token length Levers: model selection (L3), batching (L4) Where to optimize: route + batch
TRAINING COSTS: Drivers: model size, dataset size, hyperparameter tuning Levers: Spot instances, scheduled runs, checkpointing Where to optimize: instance procurement + scheduling
GPU INSTANCE COSTS: Drivers: GPU type, uptime, idle time Levers: right-sizing, scheduling, instance pools Where to optimize: M5.2 schedules + M2.4 autoscaling
EMBEDDING / VECTOR DB: Drivers: data volume, query rate Levers: storage tier, embedding compression Where to optimize: standard storage patterns (M5.1)The patterns differ; the optimization levers differ.
The biggest leverage points
For most ML-heavy workloads:
TOP LEVERAGE POINTS (impact order):
1. MODEL SELECTION (most impact) Routing simple queries to cheaper models Often 70-90% of inference cost recoverable Requires query-classification logic
2. BATCH PROCESSING (high impact) Combining inference calls (5-50x throughput) Bedrock supports batch inference at lower cost For non-real-time use cases
3. PROVISIONED THROUGHPUT TUNING (medium-high impact) Right-size committed capacity Move to on-demand when low utilization
4. TRAINING ON SPOT (medium impact) ~70-90% savings on training compute Requires checkpointing
5. GPU SCHEDULING (medium impact) Idle GPU instances cost serious money Schedule off when training not runningProvisioned-throughput waste is where the rule-detected recoverable cost concentrates. Model selection and batching usually carry the larger real-world saving, but ZopNight cannot see them: they need your application’s token mix, which never reaches the control plane.
The RC-16xx family, split
Bedrock RC-1601..1610 PT economics + orphaned artifacts RC-1615 guardrail missing PII protection RC-1630..1634 agent/model posture + idle artifacts (16 rules)
SageMaker RC-1611..1614 endpoint + HyperPod scheduling, idle endpoints, transform-job sizing RC-1616..1628 notebook/endpoint/HyperPod posture and rightsizing, e.g. RC-1621 managed spot training, RC-1628 HyperPod volume not CMK-encrypted (17 rules)
RC-1629 does not exist. Total: 33.The canonical rule names and IDs live in L2 and in the recommender rule catalog; treat that as the source of truth.
Combining patterns
TYPICAL ML COST RECOVERY:
Workload baseline: $126K/month
After applying: Model selection (route by complexity): -$80K/month (64%) Batch processing for offline tasks: -$15K/month Provisioned throughput right-sizing: -$8K/month
Result: $23K/month (-82% from baseline)
The pattern: each lever compoundsThe cumulative savings are significant for ML-heavy workloads.
2. Demo
A team’s ML cost analysis:
WORKLOAD: customer service chatbot using Bedrock Calls Claude Opus for every user query Average: 800 input + 400 output tokens Volume: 100K queries/day
CURRENT COST: 100K × (800/1M × $15 + 400/1M × $75) = 100K × ($0.012 + $0.030) = 100K × $0.042 = $4,200/day = $126K/month
ANALYSIS: Most queries are simple (greetings, status questions, simple lookups) Sonnet would handle 85% of queries adequately Haiku would handle 70% of queries adequately Opus is overkill for non-complex queries
RECOMMENDATION: route based on query complexity Simple queries (70%) → Haiku $0.80/M × 800/1M + $4/M × 400/1M = $0.0024/query 70K queries: $168/day
Medium queries (15%) → Sonnet $3/M × 800/1M + $15/M × 400/1M = $0.0084/query 15K queries: $126/day
Complex queries (15%) → Opus (unchanged) 100K × 0.15 = 15K queries $0.042/query: $630/day
REVISED COST: $168 + $126 + $630 = $924/day = $27.7K/month
SAVINGS: $126K - $28K = $98K/month (78% reduction)
EFFORT: Implement query-complexity classifier (~2 weeks) Test classification accuracy Gradual rollout Monitor quality metricsThe savings justify the engineering investment many times over.
3. Hands-on (5 min)
Identify a Bedrock workload:
□ STEP 1: Inventory Bedrock usage Workloads using Bedrock: _____ Top spender: __________
□ STEP 2: Analyze top workload Model used: __________ Avg input tokens: _____ Avg output tokens: _____ Volume per day: _____
□ STEP 3: Calculate cost Cost per call: $_____ Monthly cost: $_____
□ STEP 4: Could a cheaper model handle it? Sonnet candidate? □ Yes □ No Haiku candidate? □ Yes □ No Percentage of queries: _____%
□ STEP 5: Estimate savings Current: $_____/mo With routing: $_____/mo Savings: $_____/moA 15-minute exercise reveals the model-selection opportunity.
4. Knowledge check
Q1
A workload using Opus for all queries when most could use Haiku:
A. Significantly overspending
B. Standard practice
C. Slightly wasteful, but within tolerance
D. Required
Show answer
Correct: A. Route by complexity Opus is ~18.75x more expensive than Haiku per token. Model selection is the biggest Bedrock cost lever: often 60-80% recoverable. Model selection is the biggest Bedrock cost lever.
Q2
Bedrock charges:
A. Per request made, regardless of the token count at all times
B. Per hour
C. Per GB
D. Per input token + per output token, with rates varying by model
Show answer
Correct: D. Token-based pricing means efficiency in token count + model choice both matter. Token-based pricing per model.
Q3
GPU instances running 24/7 for training:
A. Likely wasteful: training is intermittent
B. Optimal
C. Required for ML
D. Cheaper than on-demand GPU per hour run
Show answer
Correct: A. Scheduling + Spot instances for training jobs can cut 70-90% of cost. GPU idle time is the biggest GPU-related cost waste. Scheduling + Spot for training.
5. Apply
The 16 Bedrock rules surface the control-plane patterns. Review in Recommendations → filter by RC-16xx rules. The application-layer levers in L3 and L4 will not appear there.
For your team: ML cost is often the fastest-growing line item; optimization here pays off quickly.
Related lessons
Glossary terms touched
Bedrock pricing · Model selection lever · ML cost driver · GPU scheduling