Virtual Keys: Per-Team LLM Budgets and Model Limits

A virtual key is an API key issued by an AI gateway rather than by an LLM provider. An application sends it exactly where it would send a provider key, but the gateway, not the provider, decides whether to honour it. Behind the scenes the gateway uses its own provider credential. Because the key belongs to the gateway, it can carry rules a provider key does not: a spending cap, a list of permitted models, and an owner for attribution.

Why not hand out provider keys

A provider key grants whatever the provider account allows, and every call made with it lands on one bill line. If three teams share one key, nobody can say which team spent what. If each team gets its own provider account, the organization now manages several vendor accounts per provider, and still has no cap that stops a runaway loop. Virtual keys give each consumer its own identity and limit while the provider relationship stays in one place.

What a virtual key carries in ZopNight

In ZopNight’s AI Gateway capability, which is released behind a feature flag, each virtual key has:

• A hard USD budget. The gateway enforces it; once the key has spent its budget, further requests are refused.
• A model allow-list. The key rejects any model outside the list, so a prototype key cannot call a model it was not granted.
• An owner stamp. Keys record who created them, shown as a Created by column, and rotation keeps the original creator.
• Organization scoping. Keys are stamped with the organization, so two organizations sharing a gateway fleet never see each other’s keys.

Above the keys sits an optional org budget ceiling: a hard cap on total spend across all of an organization’s keys, with each key’s own budget still applying inside it.

Alerts before the cap

A hard cap stops spend, and it also stops the feature using the key. ZopNight’s budget alerts give warning first: a key’s budget takes part in the same notifications as cloud and AI-provider budgets, firing a warning at 80 percent and an exceeded alert at 100 percent, naming the key. Delivery follows the organization’s existing budget subscriptions.

Rotation and revocation

Rotating a key mints a replacement; the new key is shown once, at creation or rotation, together with copy-paste snippets for Claude Code, the OpenAI SDK, and cURL that pin a model the key is allowed to call. Revoking a key ends its access. Store the revealed key in a secret manager, since it cannot be displayed again.

Who can manage keys

Keys are visible to the whole organization, not private to their creator: any user whose role allows it can edit, rotate, or revoke any key. Access is set in the role matrix through an AI Virtual Key capability with view, create, edit, and delete; the built-in Editor role has full control of keys, and Viewer is read-only. A role can also be scoped to one provider, for example keys that may only use Bedrock models. Rotation requires the update permission rather than create, so a create-only role cannot rotate, and therefore cannot destroy, an existing key.

Designing a key scheme

• One key per team per environment keeps attribution readable and lets a staging key have a small budget.
• Give each key the narrowest model list that works, and widen it on request.
• Set the key budget below the team’s AI budget so the alert arrives before the cap bites.

Related: AI gateways, LLM cost governance, and the step-by-step per-team LLM budgets how-to.