# Azure ML Online Deployment

> Azure ML online deployments hold a fixed set of CPU or GPU instances that bill every hour of every day, whether 1 request or 1 million arrives. ZopNight captures each deployment's instance type and count, attributes its spend from Cost Management, and flags over-provisioned inference capacity in recommendations.

Source: https://zop.dev/integrations/azure/ml-online-deployment
Updated: 2026-08-19

---

Online deployments are the provisioned instance sets behind real-time inference endpoints, billed per instance-hour continuously. Over-instanced deployments for low-traffic models burn GPU or CPU money around the clock.

## Instance-hours are the whole story

A deployment's bill is its instance count times its instance size, accruing every hour it exists. Traffic volume changes nothing about the meter: a model scoring a handful of requests a day on three GPU instances pays exactly what a saturated one pays. That makes deployment sizing the single decision that sets real-time inference cost, and makes low-traffic production models the place where the gap between provisioned and needed is widest.

## What ZopNight knows about each deployment

Discovered via the AML enricher with instance type and count, under the endpoint that routes to it. Cost Management billing attributes spend per deployment, and recommendations flag over-provisioned inference capacity: deployments whose instance count outruns any traffic they see.

## Why deployments are visibility-only for now

A deployment cannot be stopped without dropping the model out of its endpoint's traffic split, so ZopNight scopes it to discovery and cost visibility rather than scheduling. The savings lever is structural: shrink the instance count, move to a smaller SKU, or retire the deployment. All three are decisions the per-deployment cost attribution is designed to force into view.

## Where inference capacity goes to waste

Three recurring shapes: deployments provisioned "for launch traffic" that never materialized; the second half of an A/B test still holding full capacity months after the decision; and staging deployments mirroring production instance counts on models nobody calls after sign-off. GPU deployments deserve the first pass in any review, since a single over-provisioned instance there outweighs several CPU ones.

## Checking a deployment's size in ML studio

Azure ML studio → Endpoints → open the endpoint → Deployments tab shows each deployment's instance type and count next to its share of traffic. Instance counts that dwarf traffic share are the audit finding.