Real-time SageMaker endpoints not capturing inference requests and responses

A model serving blind

An endpoint without data capture answers predictions and keeps no record of what it was asked or what it said. Three capabilities disappear with that: you cannot audit an individual prediction after the fact (“why was this loan declined in March”), you cannot baseline the live traffic distribution for Model Monitor, and you cannot detect drift (the slow divergence between training data and production inputs that degrades models without a single error being thrown). Drift is the default fate of production models; capture is how you see it coming.

Confirmed-false semantics

The discoverer reads the endpoint’s data-capture state and writes an explicit boolean. Only a confirmed false on an in-service endpoint fires; a missing flag (describe failed) abstains, and endpoints outside the in-service state are excluded entirely. The rule proves the setting rather than inferring it, and carries no savings figure. Capture is an observability control, and its absence costs model quality, not compute dollars.

Inspect an endpoint’s capture config

aws sagemaker describe-endpoint-config \
  --endpoint-config-name "$(aws sagemaker describe-endpoint \
      --endpoint-name my-endpoint --query EndpointConfigName --output text)" \
  --query 'DataCaptureConfig'

A null result means no capture configuration exists at all.

Enabling it is an endpoint update, not a rebuild

Capture lives in the endpoint configuration, which is immutable, but endpoints move between configs freely. Create a new config that copies the current one plus a DataCaptureConfig (enable flag, sampling percentage, S3 destination), then update the endpoint to it; SageMaker performs a blue/green rollover with no downtime. Start sampling around 20–30% for busy endpoints (100% capture on high-throughput inference produces serious S3 volume), and treat the destination bucket as sensitive: captured payloads are production data and deserve encryption and tight access control.

Where the captured data pays off

The S3 output is directly consumable by Model Monitor: baseline from training data, schedule monitors, and alert on schema violations and distribution shift. Even without Model Monitor, captured request/response pairs are the dataset for retraining evaluation and incident forensics. The finding is medium severity because nothing is exposed. But a model you cannot audit is a governance gap that only becomes visible at the worst possible time.