SageMaker endpoints whose storage volumes use no customer managed KMS key

Encrypted, but not with a key you own

SageMaker never leaves an endpoint’s storage volume in plain text. If you do not pass a key, SageMaker encrypts the volume with a transient key and discards it immediately afterwards. That satisfies “encrypted at rest” but not the stricter requirement many security policies add: the organization must hold the key, control its key policy, see its use in CloudTrail and be able to disable it.

The KmsKeyId field of the endpoint configuration is where that customer managed key goes. It encrypts the storage volume attached to the ML compute instance that hosts the model.

Checking which key an endpoint uses

aws sagemaker describe-endpoint --endpoint-name my-endpoint \
  --query 'EndpointConfigName' --output text

aws sagemaker describe-endpoint-config --endpoint-config-name my-endpoint-config \
  --query '[KmsKeyId, ProductionVariants[].InstanceType]'

A null or empty KmsKeyId means no customer managed key was specified.

Three gates before a finding

1. The endpoint is InService.
2. The endpoint configuration ZopNight recorded shows the KMS key field present and empty.
3. The endpoint’s instance type is not one with local NVMe instance storage.

Both real-time and asynchronous inference endpoints are checked.

Why NVMe-backed endpoints are skipped

The endpoint configuration reference notes that certain Nitro-based instances include local storage encrypted by a hardware module on the instance, and that specifying KmsKeyId for those instance types makes the call fail. Flagging those endpoints would ask for a fix AWS rejects, so ZopNight leaves them out. If the key field was never captured for an endpoint, it is not flagged either.

A key-custody finding, not a cost one

The saving is $0. A customer managed key adds a small monthly KMS key charge plus request charges, and in exchange gives you revocation: disabling the key cuts off access to the data it protects.

Moving an endpoint to your own key

1. Create or choose a customer managed key, and grant the endpoint’s execution role permission to use it in the key policy.
2. Create a new endpoint configuration with the same production variants and --kms-key-id set; the key cannot be added to an existing configuration.
3. Point the endpoint at it with aws sagemaker update-endpoint --endpoint-name my-endpoint --endpoint-config-name my-endpoint-config-cmk.
4. Confirm the endpoint returns to InService, then delete the old configuration.