Outcome
By the end of this lesson, you will be able to decide between on-demand and provisioned throughput, right-size provisioned capacity, and handle idle hours without committing capacity you will want to tear down.
| Prerequisites | M2.11.L1 - L4 |
| Time | 9 minutes |
1. Concept
Bedrock will sell you model capacity two ways, and the right one depends entirely on how steady your usage is.
Pay per request and you pay only for what you send, at a higher unit price. Reserve capacity and you pay a fixed hourly amount whether you use it or not, at a much lower unit price.
ON-DEMAND: Pay per token (no commitment) Variable cost No upfront commitment
PROVISIONED THROUGHPUT: Pay per hour for guaranteed throughput No commitment (hourly), or commit 1 or 6 months Up to 50% off effective per-token rate
THE CHOICE: depends on workload's pattern + maturityThe two modes trade flexibility for cost. Provisioned wins when usage is predictable.
When provisioned wins
SCENARIO PROVISIONED OR ON-DEMAND?─────────────────────────────────────────────────────────────Predictable high-volume PROVISIONED (1-month or 6-month)Variable low-volume On-demandBurst-mode On-demandAlways-on, steady throughput PROVISIONEDNew workload, unknown load On-demand (until known)Mature workload, stable pattern PROVISIONEDLong-term commitment OK PROVISIONED 6-month (best discount)Variable traffic On-demandThe break-even point: roughly 70% utilization of provisioned throughput.
Right-sizing provisioned throughput
MEASUREMENT (the workload's actual usage): (PTs are bought in model units, not tokens/sec: convert each capacity below into model units) Average tokens/sec: 2,400 Peak tokens/sec: 9,800 Sustained periods at peak: 5% of time
PROVISIONED OPTIONS:
Option A: Provision for peak (10K tokens/sec) Capacity: 24/7 Cost: $X (highest baseline) Utilization: 24% (mostly idle) Performance: never throttled
Option B: Provision for average (2,400 tokens/sec) Your app sends bursts to the on-demand model ID Cost: ~30% of Option A Utilization: ~80% Bursts: occasional on-demand cost Total cost: typically 35% of Option A
Option C: Provision at peak × 0.7 (7K tokens/sec) On-demand for the burst above Cost: ~50% of Option A Utilization: ~65% Better burst handling than Option BOption B usually wins for variable-load workloads. Option C suits workloads where burst latency matters more than the extra provisioned cost.
Detecting over-provisioning
SIGNAL INTERPRETATION─────────────────────────────────────────────────────────────────Traffic present, but under 100,000 Over-provisioned (RC-1602,output tokens/day on more than 1 MU 30-day window) Reduce model units OR switch to on-demand
Utilization > 95% sustained Under-provisioned Consider increase to avoid throttling
Periodic peak hits ceiling Provisioned for average; OK On-demand handles peaks (if your app routes them)
Periodic drop to near-zero Idle periods Size PT to the steady base load; on-demand absorbs the peaks
Spiky utilization On-demand might be more cost-effective Variable patterns don't suit provisionedRC-1602 uses that output-token floor, not a utilization percentage. The 95% line is a rule of thumb for under-provisioning.
Idle hours: what you can and cannot do
ZopNight cannot schedule provisioned throughput. Bedrock PT is discovered and costed, but it is not a schedulable resource type, so there is no start/stop for it the way there is for a VM or a SQL warehouse.
PROVISIONED THROUGHPUT bills per hour, for every hour it exists.If a workload is quiet overnight or at weekends:
COMMITTED PT (1 or 6 months): Bills to term-end whatever you do Cannot be torn down early to save money Idle hours are paid for at the committed rate
NO-COMMITMENT PT: Can be deleted when it is no longer needed RC-1601 flags one with zero invocations (it abstains on a PT under an active commitment)
WHAT ACTUALLY AVOIDS PAYING FOR IDLE HOURS: Size the PT to the steady base load (the overnight floor) Let on-demand absorb the daytime peaks Never commit capacity you plan to tear downThe lever for idle hours is sizing and commitment choice, not a schedule.
Commitment math
1-MONTH PROVISIONED: Discount: ~20-30% Break-even: ~70% utilization Commitment: 30 days Risk: medium-low
6-MONTH PROVISIONED: Discount: ~40-50% Break-even: ~50% utilization (lower because longer commit) Commitment: 180 days Risk: medium (workload pattern must hold)
Those are the only terms: no commitment, 1 month, or6 months. There is no annual term.For mature workloads with stable patterns, 6-month wins on discount.
Hybrid pattern: provisioned + on-demand
THE BEST PATTERN for most workloads:
Provisioned: base load (e.g., 70% of peak) On-demand: bursts above provisioned capacity (your app routes them)
EXAMPLE: Workload peaks 10K tokens/sec; sustains 4K
Provision 7K tokens/sec Cost: provisioned hourly rate
Burst above 7K: on-demand Pay for the burst tokens Most days: minimal on-demand cost
TOTAL COST: significantly lower than provisioning for peakPERFORMANCE: handles peaks without throttlingThe hybrid pattern is the standard for variable-but-predictable workloads.
Decision framework
QUESTIONS TO ANSWER for any Bedrock workload:
1. Is the workload mature? (running stable >3 months) Yes → consider provisioned No → stay on-demand
2. Is utilization predictable? Yes → provisioned at average + on-demand for bursts No → on-demand
3. Does it have idle periods (overnight/weekend)? Yes → size PT to the base load; on-demand for peaks No → provisioned sized to steady throughput
4. Can you commit 6 months? Yes → 6-month for best discount No → 1-month or on-demand
5. Is current cost a concern? Yes → optimize aggressively (right-size + hybrid) No → accept current patternThe framework guides the decision per workload.
2. Demo
A team’s provisioned throughput optimization:
WORKLOAD: customer support chatbot Mature pattern over 6 months Average: 4,000 tokens/sec Peak: 7,500 tokens/sec Distribution: 95% of time within 2.5x of average Off-hours: significant drop in volume (but not zero)
OPTION A (CURRENT): on-demand Cost: ~$28,000/month Predictable; variable; no commitment
OPTION B: provisioned at 4,500 tokens/sec; on-demand for bursts Cost: ~$15,000/month provisioned + ~$2,000/month on-demand bursts Total: $17,000/month Savings: ~$11,000/month (39%)
OPTION C: 6-month commitment on Option B's capacity Provisioned 4,500 tokens/sec, committed for 180 days Additional discount on provisioned hours Total: ~$13,000/month ($11,000 provisioned + $2,000 bursts) Savings: ~$15,000/month (54%) Trade-off: quiet overnight hours are still paid for
NOT AN OPTION: turning the PT off overnight ZopNight cannot schedule provisioned throughput A committed PT bills to term-end whatever you do
DECISION: Option C for the mature, stable workload pattern
IMPLEMENTATION (4 weeks): Week 1: validate utilization measurements Week 2: run a no-commitment PT at 4,500 tokens/sec Week 3: buy the 6-month PT at the validated size; delete the no-commitment PT Week 4: monitor + validate cost projections
OUTCOMES (1 month in): Actual cost: $13,300/mo (within 3% of projection) Quality: unchanged Performance: peaks handled by on-demand burst Engineer satisfaction: highProvisioned at the base load + on-demand for bursts, committed only once the size is proven, is often the optimal pattern for mature workloads.
3. Hands-on (5 min)
Optimize your provisioned throughput:
□ STEP 1: Inventory provisioned throughput Workload: __________ Current capacity: _____ tokens/sec Current cost: $_____/mo
□ STEP 2: Measure actual usage Average: _____ tokens/sec Peak: _____ tokens/sec Utilization: ____%
□ STEP 3: Right-sizing options Reduce to average + on-demand bursts: $_____/mo Size to steady base load + on-demand peaks: $_____/mo Commit 6-month (only at a proven size): $_____/mo
□ STEP 4: Pick best option Recommended: __________ Projected savings: $_____/mo
□ STEP 5: Plan Implementation effort: ___ weeks Risk: __________ Monitoring plan: __________A 15-minute optimization exercise per workload.
4. Knowledge check
Q1
A workload has 25% sustained utilization on its provisioned throughput. The decision:
A. Increase throughput
B. Investigate over-provisioning: reduce throughput or switch to on-demand
C. Commit for a longer term to lower the effective hourly rate further
D. Keep as is
Show answer
Correct: B. At 25% utilization the PT is well below the 70% break-even. ZopNight’s RC-1602 (PT Underutilized) flags a PT with traffic but under 100,000 output tokens/day on more than one model unit; RC-1601 fires only on zero invocations.
Q2
Provisioned throughput discount vs on-demand:
A. Same price
B. 90% discount
C. A flat 75% off, on any commitment length
D. ~20-50% discount depending on commit length
Show answer
Correct: D. 1-month: 20-30%; 6-month: 40-50%. Break-even at 50-70% utilization. You trade flexibility for cost.
Q3
A mature workload is busy by day and quiet at night. How do you avoid paying for idle overnight PT capacity?
A. Have ZopNight schedule the PT off overnight
B. Size the PT to base load; on-demand takes peaks
C. Buy a 6-month term, then delete it at night
D. Move the whole workload to on-demand at night
Show answer
Correct: B. ZopNight cannot schedule provisioned throughput, and a committed PT bills to term-end whatever you do. Sizing the PT to the steady base load and letting on-demand absorb the peaks keeps the paid-for capacity busy around the clock.
5. Apply
RC-1601 (idle) and RC-1602 (underutilized) surface provisioned-throughput optimization; RC-1610 catches a PT still pinned to a deprecated model generation. Customer adjusts in the Bedrock console.
For your team: right-size provisioned throughput to the steady base load; let on-demand absorb peaks; commit 6-month only for mature workloads at a proven size.
Module quiz
Complete M2.11 → 10-question module quiz.
Related lessons
- L1: ML cost landscape
- L2: Bedrock rules
- L3: Model selection
- L4: Batch processing
- T6: AI-powered cloud ops