Outcome
By the end of this lesson, you will be able to map ML/Bedrock cost surface to specific optimization levers, identify the major cost drivers, and recognize which Bedrock rule applies to each pattern.
| Prerequisites | M2.1 (rule library) |
| Time | 9 minutes |
1. Concept
Machine learning spending comes in three shapes, and each is reduced differently.
Running a model is charged per request. Training a model is charged by the hour. The machines underneath, usually GPUs, are charged for as long as they exist, whether or not anything is using them.
COST DRIVER TYPICAL % OPTIMIZATION──────────────────────────────────────────────────────────────Bedrock model inference 30-60% Model selection, batch processingFoundation model training 20-40% Spot instances, scheduled runsGPU instance hours 15-30% Right-sizing, schedulingVector DB / embedding store 5-15% Standard storage optimizationSageMaker endpoints 5-20% Right-sizing endpoints; serverlessML cost optimization requires understanding which lever applies where.
Bedrock-specific patterns
Bedrock charges per input token + per output token by model. Different models have wildly different costs:
ILLUSTRATIVE PRICES ONLY, NOT CURRENT BEDROCK LIST PRICES
MODEL INPUT $/M TOKENS OUTPUT $/M TOKENS RELATIVE COST─────────────────────────────────────────────────────────────────────────Claude Opus $15.00 $75.00 1.0x (baseline)Claude Sonnet $3.00 $15.00 0.2x (5x cheaper)Claude Haiku $0.80 $4.00 ~0.053x (~19x cheaper)Titan, Llama, check the Bedrock pricing pageMistralThese prices are illustrative. They show the ratio between a large, a mid-size and a small model, and they keep the arithmetic in this module consistent. They are not current Bedrock list prices, and real ratios change as new models ship. Check the Bedrock pricing page before using any number here for a real decision.
At these illustrative prices, a workload using Opus when Haiku would suffice pays about 19x what it needs to. Real ratios differ from model to model, but the gap between the largest and smallest model is still the biggest lever in this module.
Detection signals
ZopNight detects ML cost patterns:
IDLE BEDROCK PROVISIONED THROUGHPUT: Pattern: capacity reserved but unused Detection: Invocations series absent over a 30-day lookback Rule: RC-1601 (PT Idle): full recovered cost Action: migrate to on-demand, delete the PT
PROVISIONED THROUGHPUT ON A DEPRECATED MODEL: Pattern: PT still pinned to an older in-family generation Detection: modelArn in the deprecated set + a cheaper successor Rule: RC-1610 (PT on Older Model Generation) Action: recreate the PT on the successor, migrate, delete
ORPHANED BEDROCK ARTIFACTS: Pattern: custom models, KBs, imported models nothing references Detection: no invocations / no referencing deployment Rule: RC-1604, RC-1605, RC-1606 Action: guided delete
TRAINING JOBS ON ON-DEMAND: Pattern: training using premium pricing Detection: training workload type + on-demand tier Rule: RC-1621 (SageMaker managed spot training) Action: switch to managed Spot
HIGH INFERENCE COST PER TOKEN: Pattern: high-cost model used for simple queries Detection: NOT DETECTED BY ANY RULE: the Bedrock family reads the control plane, not your prompts or token mix. This is a real lever (L3) with no rule behind it. Action: route by complexity, in your applicationThe 16 Bedrock rules span RC-1601..1610 plus RC-1615 and RC-1630..1634. They sit inside a broader ML family of about 33 rules; the other 17 are SageMaker, RC-1611..1614 and RC-1616..1628. See L2 for the authoritative per-rule list.
Cost drivers per pattern
INFERENCE COSTS (Bedrock or self-hosted): Drivers: model choice, request volume, token length Levers: model selection (L3), batching (L4) Where to optimize: route + batch
TRAINING COSTS: Drivers: model size, dataset size, hyperparameter tuning Levers: Spot instances, scheduled runs, checkpointing Where to optimize: instance procurement + scheduling
GPU INSTANCE COSTS: Drivers: GPU type, uptime, idle time Levers: right-sizing, scheduling, instance pools Where to optimize: M3.2 schedules + M2.4 autoscaling
EMBEDDING / VECTOR DB: Drivers: data volume, query rate Levers: storage tier, embedding compression Where to optimize: standard storage patterns (M3.1)The patterns differ; the optimization levers differ.
The biggest leverage points
For most ML-heavy workloads:
TOP LEVERAGE POINTS (impact order):
1. MODEL SELECTION (most impact) Routing simple queries to cheaper models Often 70-90% of inference cost recoverable Requires query-classification logic
2. BATCH PROCESSING (high impact) Combining inference calls (5-50x throughput) Bedrock supports batch inference at lower cost For non-real-time use cases
3. PROVISIONED THROUGHPUT TUNING (medium-high impact) Right-size committed capacity Move to on-demand when low utilization
4. TRAINING ON SPOT (medium impact) ~70-90% savings on training compute Requires checkpointing
5. GPU SCHEDULING (medium impact) Idle GPU instances cost serious money Schedule off when training not runningProvisioned-throughput waste is where the rule-detected recoverable cost concentrates. Model selection and batching usually carry the larger real-world saving, but ZopNight cannot see them: they need your application’s token mix, which never reaches the control plane.
The RC-16xx family, split
Bedrock RC-1601..1610 PT economics + orphaned artifacts RC-1615 guardrail missing PII protection RC-1630..1634 agent/model posture + idle artifacts (16 rules)
SageMaker RC-1611..1614 endpoint + HyperPod scheduling, idle endpoints, transform-job sizing RC-1616..1628 notebook/endpoint/HyperPod posture and rightsizing, e.g. RC-1621 managed spot training, RC-1628 HyperPod volume not CMK-encrypted (17 rules)
RC-1629 does not exist. Total: 33.The canonical rule names and IDs live in L2 and in the recommender rule catalog; treat that as the source of truth.
Combining patterns
TYPICAL ML COST RECOVERY:
Workload baseline: $126K/month
After applying: Model selection (route by complexity): -$80K/month (64%) Batch processing for offline tasks: -$15K/month Provisioned throughput right-sizing: -$8K/month
Result: $23K/month (-82% from baseline)
The pattern: each lever compoundsThe cumulative savings are significant for ML-heavy workloads.
2. Demo
A team’s ML cost analysis:
WORKLOAD: customer service chatbot using Bedrock Calls Claude Opus for every user query Average: 800 input + 400 output tokens Volume: 100K queries/day
CURRENT COST (illustrative prices from the table above): 100K × (800/1M × $15 + 400/1M × $75) = 100K × ($0.012 + $0.030) = 100K × $0.042 = $4,200/day = $126K/month
ANALYSIS: Most queries are simple (greetings, status questions, simple lookups) Sonnet would handle 85% of queries adequately Haiku would handle 70% of queries adequately Opus is overkill for non-complex queries
RECOMMENDATION: route based on query complexity Simple queries (70%) → Haiku $0.80/M × 800/1M + $4/M × 400/1M = $0.00224/query 70K queries: $156.80/day
Medium queries (15%) → Sonnet $3/M × 800/1M + $15/M × 400/1M = $0.0084/query 15K queries: $126/day
Complex queries (15%) → Opus (unchanged) 100K × 0.15 = 15K queries $0.042/query: $630/day
REVISED COST: $156.80 + $126 + $630 = $912.80/day = $27.4K/month
SAVINGS: $126K - $27.4K = $98.6K/month (78% reduction)
EFFORT: Implement query-complexity classifier (~2 weeks) Test classification accuracy Gradual rollout Monitor quality metricsThe savings justify the engineering investment many times over.
3. Hands-on (5 min)
Identify a Bedrock workload:
□ STEP 1: Inventory Bedrock usage Workloads using Bedrock: _____ Top spender: __________
□ STEP 2: Analyze top workload Model used: __________ Avg input tokens: _____ Avg output tokens: _____ Volume per day: _____
□ STEP 3: Calculate cost Cost per call: $_____ Monthly cost: $_____
□ STEP 4: Could a cheaper model handle it? Sonnet candidate? □ Yes □ No Haiku candidate? □ Yes □ No Percentage of queries: _____%
□ STEP 5: Estimate savings Current: $_____/mo With routing: $_____/mo Savings: $_____/moA 15-minute exercise reveals the model-selection opportunity.
4. Knowledge check
Q1
A workload using Opus for all queries when most could use Haiku:
A. Significantly overspending
B. Standard practice
C. Slightly wasteful, but within tolerance
D. Required
Show answer
Correct: A. At this module’s illustrative prices Opus is ~18.75x more expensive than Haiku per token, so routing by complexity is the fix. Model selection is the biggest Bedrock cost lever.
Q2
Bedrock charges:
A. Per request made, regardless of the token count at all times
B. Per hour
C. Per GB
D. Per input token + per output token, with rates varying by model
Show answer
Correct: D. Pricing is per token, per model, so token count and model choice both matter.
Q3
GPU instances running 24/7 for training:
A. Likely wasteful: training is intermittent
B. Optimal
C. Required for ML
D. Cheaper than on-demand GPU per hour run
Show answer
Correct: A. GPU idle time is the biggest GPU-related cost waste, and scheduling plus Spot instances for training jobs cut a large share of it.
5. Apply
The 16 Bedrock rules surface the control-plane patterns. Review in Recommendations → filter by RC-16xx rules. The application-layer levers in L3 and L4 will not appear there.
For your team: ML cost is often the fastest-growing line item; optimization here pays off quickly.