Security and data protection for inference routers
Security for inference routers follows the AWS shared responsibility model described in Configure security in Amazon SageMaker AI. This topic covers what is specific to inference routers.
IAM resources and condition keys
An inference router has the following ARN format:
arn:partition:sagemaker:region:account-id:inference-router/inference-router-name
You can use aws:ResourceTag/${TagKey} to control access to
InvokeInferenceRouter,
InvokeInferenceRouterAsync,
DescribeInferenceRouter, UpdateInferenceRouter, and
DeleteInferenceRouter, and aws:RequestTag/${TagKey} and
aws:TagKeys to control CreateInferenceRouter. The
sagemaker:SourceInferenceRouterArn condition key (ARN type) applies to
sagemaker:InvokeEndpoint and
sagemaker:InvokeEndpointAsync. It contains the ARN of the router that
sent the request, and is absent on direct calls. For the actions, resources, and condition keys
for inference routers, see Actions,
resources, and condition keys for Amazon SageMaker AI in the Service
Authorization Reference.
How the router authorizes requests to destinations
The router calls each destination with your caller's own credentials, so your callers
can reach only the destinations that they're allowed to invoke. The router doesn't
assume a role or share credentials between accounts. SageMaker AI sets
sagemaker:SourceInferenceRouterArn only on requests that an inference
router sends; a caller can't set it. To limit which routers can use your endpoint, add
this condition to your IAM policies or endpoint policy with the router's full ARN.
Endpoint policies require it on statements that grant another account invoke access.
For more information, see Route requests to endpoints in other AWS accounts.
Data protection
A request is processed in the router's AWS Region and in the Region of the destination that serves it. To keep requests within allowed Regions, see Keep requests within allowed Regions. For general guidance, see Data Protection in Amazon SageMaker AI.
The router stores the following data:
-
Your router configuration (name, routing targets, destination ARNs, and asynchronous output settings) and your endpoint policies, until you delete them.
-
For each asynchronous request, a record of the request settings: the Amazon S3 input, output, and failure locations, the AWS KMS key ID, content type,
Accept, inference ID, custom attributes, timeouts, and the destinations tried. Credentials stored for an asynchronous request are encrypted and deleted when the request ends. The router deletes the record 7 days after it accepts the request.
The router doesn't store request or response payloads. For asynchronous requests, inputs and outputs stay in the Amazon S3 locations that you specify.
Keep requests within allowed Regions
A routing target can send any request to any of its destinations. To keep requests within a set of allowed Regions, such as the EU Regions:
-
Create one router for each set of allowed Regions, in one of those Regions. Add only destinations in those Regions.
-
Allow each caller to invoke only the routers for its Regions. In the caller's IAM policy, list the router ARNs in the
Resourceelement ofsagemaker:InvokeInferenceRouterandsagemaker:InvokeInferenceRouterAsync. -
As a guardrail, use a service control policy (SCP) that denies
sagemaker:InvokeEndpointandsagemaker:InvokeEndpointAsyncoutside the allowed Regions. The router calls each destination with your caller's credentials, so the SCP applies to requests that the router sends. A deny onsagemaker:InvokeInferenceRouterdoesn't do this, becauseaws:RequestedRegionfor that action is the router's Region. The following SCP also denies direct calls to endpoints outside the allowed Regions.{ "Version": "2012-10-17", "Statement": [ { "Effect": "Deny", "Action": ["sagemaker:InvokeEndpoint", "sagemaker:InvokeEndpointAsync"], "Resource": "*", "Condition": { "StringNotEquals": { "aws:RequestedRegion": ["eu-west-1", "eu-central-1"] } } } ] } -
To limit which routers can send requests to an endpoint, add the
sagemaker:SourceInferenceRouterArncondition to the endpoint's policies. For more information, see How the router authorizes requests to destinations.
For asynchronous requests, the input and output buckets are in the router's Region, so choose a router Region within the allowed set.
When a destination in your own account denies a real-time request, the router
retries it once on another destination. If that destination also denies it, you get
an AccessDeniedException. When a destination in another account denies
a request, the caller gets ServiceUnavailableException (503) instead
of AccessDeniedException, so that the caller can't tell whether the
endpoint exists. This applies to both real-time and asynchronous requests.
When a destination denies an asynchronous
request, the router tries the other destinations. You get an
AccessDeniedException (same account) or
ServiceUnavailableException (other account) only if every destination
denies the request.
Encryption at rest
SageMaker AI encrypts the router configuration, endpoint policies, and asynchronous request records at rest with keys that AWS owns and manages. You can't use a customer managed key for this data.
For asynchronous routing targets, if you set KmsKeyId in
OutputConfig, output and failure objects are encrypted with your
customer managed key. Otherwise, output uses the default encryption of the output
bucket, and failure objects use SSE-KMS with the AWS managed key for Amazon S3
(aws/s3). For more information, see Protect Data at Rest Using Encryption.
Encryption in transit
Calls to an inference router use HTTPS with TLS 1.2 or later. For the hop from the router to a destination in another Region, see Cross-Region requests. For more information, see Protecting Data in Transit with Encryption.
Cross-Region requests
When an inference router sends a request to a destination in another Region, it forwards the request over TLS 1.2 or later to the SageMaker AI runtime endpoint in that Region, signed with your credentials. The request is authorized in the destination Region against your IAM policies, the endpoint's resource-based policy, and your service control policies. Traffic between Regions stays on the AWS network. If you call the router through an interface VPC endpoint (AWS PrivateLink), the router reaches every destination, including one in its own Region, through the public SageMaker AI runtime endpoint of the destination's Region, not through PrivateLink.
Invoke inference routers through an interface VPC endpoint
You can call InvokeInferenceRouter and
InvokeInferenceRouterAsync from your Amazon VPC through an interface VPC
endpoint (AWS PrivateLink), which keeps traffic between your Amazon VPC and the router off the
public internet. Create the endpoint in the router's AWS Region with the following
service name:
com.amazonaws.region.sagemaker.inferencerouter-runtime
For steps, see Access an AWS service using an interface VPC endpoint in the AWS PrivateLink Guide.
-
If you turn on private DNS, the router's default hostname,
inferencerouter-runtime.sagemaker., resolves to your interface VPC endpoint, and you don't have to change your application.region.amazonaws.com -
The endpoint supports IPv4 only.
-
You can attach a VPC endpoint policy to control which principals can invoke which routers through the endpoint. The following VPC endpoint policy allows one router to be invoked through the endpoint.
{ "Statement": [ { "Effect": "Allow", "Principal": "*", "Action": [ "sagemaker:InvokeInferenceRouter", "sagemaker:InvokeInferenceRouterAsync" ], "Resource": "arn:aws:sagemaker:us-west-2:111122223333:inference-router/my-router" } ] } -
The interface VPC endpoint covers the call to the router only. The router reaches each destination, in any Region, through the public SageMaker AI runtime endpoint of that Region, not through an interface VPC endpoint. For more information, see Cross-Region requests.