Route requests to endpoints in other AWS accounts
With an inference router, you can send requests to an endpoint or inference component that another AWS account owns. Both accounts must allow it:
-
The endpoint owner attaches an endpoint policy (a resource-based policy) that allows the router's account to invoke the endpoint through that router.
-
The router owner allows its callers to invoke the router and the destination, and adds the destination to a routing target.
The caller's identity needs permission in both accounts. For the reason, see How the router authorizes requests to destinations.
Example endpoint policy
The endpoint owner (account 444455556666) runs the following
command in the endpoint's AWS Region. The first statement lets account
111122223333 invoke the endpoint only through
my-router. The second lets that account describe the
endpoint, so SageMaker AI can show its RoutingStatus and check its mode.
aws sagemaker put-endpoint-policy \ --regioneu-west-1\ --endpoint-nameshared-endpoint\ --resource-policy '{ "Version": "2012-10-17", "Statement": [ { "Sid": "InvokeThroughRouter", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::111122223333:root" }, "Action": "sagemaker:InvokeEndpoint", "Resource": "arn:aws:sagemaker:eu-west-1:444455556666:endpoint/shared-endpoint", "Condition": { "ArnEquals": { "sagemaker:SourceInferenceRouterArn": "arn:aws:sagemaker:us-west-2:111122223333:inference-router/my-router" } } }, { "Sid": "DescribeForRouterAccount", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::111122223333:root" }, "Action": "sagemaker:DescribeEndpoint", "Resource": "arn:aws:sagemaker:eu-west-1:444455556666:endpoint/shared-endpoint" } ] }'
In the command, replace the following values:
-
444455556666– the account ID of the endpoint owner. -
111122223333– the account ID of the router owner. -
shared-endpoint– the name of your endpoint. -
my-router– the name of the inference router. -
eu-west-1– the endpoint's Region. -
us-west-2– the router's Region.
To share an inference component, add the inference component ARN to the
Resource of both statements, together with the endpoint ARN, and add
sagemaker:DescribeInferenceComponent to the second statement.
For an asynchronous endpoint, allow sagemaker:InvokeEndpointAsync in the
first statement instead of sagemaker:InvokeEndpoint.
Allow callers in the router account
As the router owner (account 111122223333), attach a policy
like the following to the IAM identities that invoke the router. It allows the caller
to invoke my-router, and to invoke and describe the shared
endpoint in account 444455556666.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "InvokeRouter", "Effect": "Allow", "Action": "sagemaker:InvokeInferenceRouter", "Resource": "arn:aws:sagemaker:us-west-2:111122223333:inference-router/my-router" }, { "Sid": "InvokeAndDescribeSharedEndpoint", "Effect": "Allow", "Action": [ "sagemaker:InvokeEndpoint", "sagemaker:DescribeEndpoint" ], "Resource": "arn:aws:sagemaker:eu-west-1:444455556666:endpoint/shared-endpoint" } ] }
For an asynchronous routing target, allow
sagemaker:InvokeInferenceRouterAsync and
sagemaker:InvokeEndpointAsync instead of
sagemaker:InvokeInferenceRouter and
sagemaker:InvokeEndpoint, and add the permissions in Permissions for asynchronous requests.
Then add the shared endpoint's ARN as a destination in a routing target with
CreateInferenceRouter or UpdateInferenceRouter, as shown
in Get started with inference routers. The following example shows the
destination entry to add to the routing target.
{ "ResourceArn": "arn:aws:sagemaker:eu-west-1:444455556666:endpoint/shared-endpoint" }
Endpoint policy rules
SageMaker AI rejects an endpoint policy that doesn't follow these rules.
-
Versionmust be2012-10-17. The policy must contain only ASCII characters and must be within the size quota in Quotas and limits. -
Effectmust beAllow.NotPrincipal,NotAction, andNotResourcearen't supported. -
Principalmust name each AWS account (account ID orrootARN) or IAM role explicitly in theAWSform. Wildcard (*), service, federated, and IAM group principals aren't supported. Principals must exist when you put the policy. -
Actioncan besagemaker:InvokeEndpoint,sagemaker:InvokeEndpointAsync,sagemaker:InvokeEndpointWithResponseStream,sagemaker:DescribeEndpoint, andsagemaker:DescribeInferenceComponent. Wildcards aren't supported. Allowingsagemaker:InvokeEndpointWithResponseStreamhas no effect for inference routers. -
Resourcemust be the endpoint's ARN or the ARNs of inference components in the same account and Region. -
Conditionoperators can beArnEquals,ArnLike,StringEquals, andStringLike. Condition keys can besagemaker:SourceInferenceRouterArn,sagemaker:TargetModel, andsagemaker:TargetContainerHostname. -
Conditionmust include asagemaker:SourceInferenceRouterArnkey that names an inference router in every statement that allows another account to invoke the endpoint. We recommendArnEqualswith the full router ARN. -
The
sagemaker:SourceInferenceRouterArnvalue must be an inference router ARN in the endpoint's partition, with a 12-digit account ID. Wildcards (*or?) aren't allowed in the partition, service, or account ID. WithArnLikeorStringLike, you can use wildcards in the Region and the router name. WithArnEqualsorStringEquals, wildcards aren't allowed. -
A statement that allows
sagemaker:DescribeEndpointorsagemaker:DescribeInferenceComponentcan't have aCondition. Grant these actions in their own statement, as in the example above.
You can't put an endpoint policy on an endpoint that is being deleted or that has
failed (status Deleting or Failed).
To manage the policy, use PutEndpointPolicy,
GetEndpointPolicy, and DeleteEndpointPolicy.
PutEndpointPolicy replaces the whole policy. To change one statement,
get the current policy, edit it, and put it back. GetEndpointPolicy returns
a ValidationException if the endpoint has no policy.
Revoke access
To revoke one account, put the policy again without that account's statements. To
revoke every account, run delete-endpoint-policy. Changes typically take
effect within 30 seconds, and can take up to 5 minutes.
After access is revoked, the destination denies requests from the router, and your
callers can get ServiceUnavailableException (503). If sagemaker:DescribeEndpoint is
also removed, the destination's RoutingStatus becomes Unknown.
Neither account is notified. If you own the router, remove the destination with
UpdateInferenceRouter. If you own the endpoint, tell the router owner
that you revoked access.
If you delete an endpoint and create a new one with the same name, the new endpoint has no policy. As the endpoint owner, put the policy again on the new endpoint.
Set up cross-account access with AWS CloudFormation
You can create inference routers and endpoint policies with the AWS CloudFormation resource
types AWS::SageMaker::InferenceRouter and
AWS::SageMaker::EndpointPolicy. For
AWS::SageMaker::EndpointPolicy, the endpoint must be in the same
AWS account and AWS Region as the stack, so a router and an endpoint policy in
different accounts need a stack in each account. For the properties of each type, see AWS::SageMaker::InferenceRouter and AWS::SageMaker::EndpointPolicy in the CloudFormation Template
Reference.
The endpoint owner (account 444455556666) deploys the
following template in eu-west-1 first. It allows account
111122223333 to invoke the endpoint only through
my-router.
AWSTemplateFormatVersion: "2010-09-09" Resources: SharedEndpointPolicy: Type: AWS::SageMaker::EndpointPolicy Properties: EndpointArn: arn:aws:sagemaker:eu-west-1:444455556666:endpoint/shared-endpointPolicyDocument: Version: "2012-10-17" Statement: - Sid: InvokeThroughRouter Effect: Allow Principal: AWS: arn:aws:iam::111122223333:root Action: sagemaker:InvokeEndpoint Resource: arn:aws:sagemaker:eu-west-1:444455556666:endpoint/shared-endpointCondition: ArnEquals: sagemaker:SourceInferenceRouterArn: arn:aws:sagemaker:us-west-2:111122223333:inference-router/my-router- Sid: DescribeForRouterAccount Effect: Allow Principal: AWS: arn:aws:iam::111122223333:root Action: sagemaker:DescribeEndpoint Resource: arn:aws:sagemaker:eu-west-1:444455556666:endpoint/shared-endpoint
To avoid drift, specify principals as full ARNs, as in this example. For the policy rules, see Endpoint policy rules.
The router owner (account 111122223333) then deploys the
following template in us-west-2. It creates a router with one routing
target and two destinations: an endpoint in the router owner's account and the shared
endpoint.
AWSTemplateFormatVersion: "2010-09-09" Resources: MyRouter: Type: AWS::SageMaker::InferenceRouter Properties: InferenceRouterName:my-routerRoutingTargets: - Name:my-modelDestinations: - ResourceArn: arn:aws:sagemaker:us-west-2:111122223333:endpoint/my-endpoint-usw2- ResourceArn: arn:aws:sagemaker:eu-west-1:444455556666:endpoint/shared-endpoint
In the templates, replace the following values:
-
444455556666– the account ID of the endpoint owner. -
111122223333– the account ID of the router owner. -
shared-endpointandmy-endpoint-usw2– the names of your endpoints. -
my-router– the name of the inference router. Specify a name, so that the endpoint policy can refer to the router's ARN. -
my-model– the name of the routing target that callers pass on each request. -
eu-west-1– the endpoint's Region. -
us-west-2– the router's Region.
Changing InferenceRouterName replaces the router and changes its ARN, so
also update the endpoint policies and IAM policies that refer to it.