OpenAI
How does ZopNight track OpenAI spend?
ZopNight registers 5 OpenAI models by default through the AI Gateway: gpt-4o and gpt-4-turbo as strong tiers, gpt-4o-mini as the cheap tier, and o1 with o1-mini for reasoning. Spend is attributed per request, per team and per key, against hard USD budgets you set.
Models registered by default
| Model | Tier | Maps to |
|---|---|---|
gpt-4o | strong | openai/gpt-4o |
gpt-4o-mini | cheap | openai/gpt-4o-mini |
o1 | strong | openai/o1 |
o1-mini | cheap | openai/o1-mini |
gpt-4-turbo | strong | openai/gpt-4-turbo |
GPT-4o, GPT-4 Turbo and the o1 reasoning pair
Five, split by tier so routing has somewhere to send each class of request:
- Strong:
gpt-4o,gpt-4-turbo,o1 - Cheap:
gpt-4o-mini,o1-mini
The o1 pair are reasoning models, and worth routing deliberately: they are the expensive
option for problems that need them and a poor default for everything else.
OpenAI calls do not traverse ZopNight
ZopNight never sits in your request path. The gateway runs on hosted LiteLLM; ZopNight configures it, meters it and enforces the budget, but your prompts and completions do not traverse our infrastructure. That distinction is why per-request cost attribution is possible without us reading request bodies.
Virtual OpenAI keys with hard USD caps
Each team gets a virtual key with a hard USD budget and its own model allow-list. When a key hits its cap it stops working. This is a limit, not an alert. Keys rotate and revoke independently, so a leaked key is contained to one team’s budget rather than the org’s.
Cost-saver, Balanced and Quality-first for OpenAI
Three postures decide which model serves a request: Cost-saver, Balanced and Quality-first. Requests are classified into four semantic categories (coding, analysis and reasoning, extraction and classification, chit-chat) and routed accordingly, so cheap models take the traffic that does not need an expensive one.
A semantic cache sits in front, with strictness settings from strict (0.98 similarity) to loose (0.90). On repetitive workloads the cache is usually a larger saving than the routing itself.
OpenAI: common questions
Which OpenAI models are registered by default?
Five: gpt-4o, gpt-4-turbo and o1 in the strong tier, gpt-4o-mini and o1-mini in the cheap tier.
Should I route everything to o1?
No. The o1 pair are reasoning models, which makes them the expensive right answer for problems that need them and a poor default for everything else. Route them deliberately rather than making one the org default.
What happens when a team's OpenAI key hits its budget?
The key stops working. Virtual keys carry a hard USD cap rather than an alert, each with its own model allow-list, so a leaked key is contained to one team's budget instead of the org's.