<a id="cli-aws-sagemaker-create-inference-router"></a>

# create-inference-router

## Description

Creates an inference router. An inference router is a Regional SageMaker AI resource that sends each inference request to one of the endpoints or inference components that you list for a routing target. These destinations can be in any Amazon Web Services account, and in any Amazon Web Services Region in the same partition as the router.

The router’s status is `Creating` until it’s ready to use. SageMaker AI checks that each endpoint is a real-time or asynchronous endpoint, as the routing target requires, and rejects one of the wrong type. It checks endpoints in other accounts with your credentials, so the endpoint policy must allow `sagemaker:DescribeEndpoint` . SageMaker AI accepts an endpoint that it can’t check, for example one that doesn’t exist. Requests to it fail if it’s the wrong type or doesn’t exist. To see the status of each destination, call [DescribeInferenceRouter](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeInferenceRouter.html) and see the `RoutingStatus` of each destination.

To invoke the router, use `InvokeInferenceRouter` or `InvokeInferenceRouterAsync` . For information about inference routers, see [Route requests across Regions with inference routers](https://docs.aws.amazon.com/sagemaker/latest/dg/inference-router.html) .

See also: [AWS API Documentation](https://docs.aws.amazon.com/goto/WebAPI/sagemaker-2017-07-24/CreateInferenceRouter)

## Synopsis

```
  create-inference-router
--inference-router-name <value>
--routing-targets <value>
[--client-token <value>]
[--tags <value>]
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--debug]
[--endpoint-url <value>]
[--no-verify-ssl]
[--no-paginate]
[--output <value>]
[--query <value>]
[--profile <value>]
[--region <value>]
[--version <value>]
[--color <value>]
[--no-sign-request]
[--ca-bundle <value>]
[--cli-read-timeout <value>]
[--cli-connect-timeout <value>]
[--cli-binary-format <value>]
[--no-cli-pager]
[--cli-auto-prompt]
[--no-cli-auto-prompt]
[--cli-error-format <value>]
[--help]
```

## Options

`--inference-router-name` (string) [required]

> The name of the inference router. The name must be unique within your account in the current Amazon Web Services Region. The name is case sensitive.

> Constraints:

> * min: `1`
> * max: `63`
> * pattern: `[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}`

`--routing-targets` (list) [required]

> The routing targets of the inference router. Each routing target is a model that callers name when they invoke the router, and the destinations that serve it.

> Constraints:

> * min: `1`
> * max: `20`

> (structure)

> > A model that an inference router serves, and the destinations that can serve it. When you invoke the router, you name the routing target, and the router sends the request to one of its destinations.

> > Every destination in a routing target must serve the same model, with the same request and response format.

> > Name -> (string) [required]

> > > The name of the routing target. Callers specify this name in the `RoutingTargetName` parameter when they invoke the inference router. The name must be unique within the router.

> > > Constraints:

> > > * min: `1`
> > > * max: `63`
> > > * pattern: `[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}`

> > Destinations -> (list) [required]

> > > The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.

> > > In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow `sagemaker:DescribeEndpoint` . If an endpoint can’t be checked, for example because it isn’t found or can’t be read in time, it’s accepted, and requests to it fail if it’s the wrong mode.

> > > Constraints:

> > > * min: `1`
> > > * max: `10`

> > > (structure)

> > > > An endpoint or inference component that serves a routing target.

> > > > ResourceArn -> (string) [required]

> > > > > The Amazon Resource Name (ARN) of the endpoint or inference component. The destination can be in any account, and in any Amazon Web Services Region in the same partition as the inference router. To use a destination in another account, its owner must allow your account to invoke it through your router, in an endpoint policy.

> > > > > Constraints:

> > > > > * min: `20`
> > > > > * max: `2048`
> > > > > * pattern: `arn:aws[a-z-]*:sagemaker:[a-z0-9-]+:[0-9]{12}:(endpoint|inference-component)/[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}`

> > > > RoutingStatus -> (string)

> > > > > Whether the inference router can send requests to the destination, as checked when you called `DescribeInferenceRouter` . This value isn’t the destination’s own status. SageMaker AI ignores this field in `CreateInferenceRouter` and `UpdateInferenceRouter` requests.

> > > > > > Routable

> > > > > The destination is serving and the router can send it requests. A routable destination can still be overloaded.

> > > > > > Unroutable

> > > > > The destination can’t serve requests. For example, it’s being created or has failed, or it has no capacity and no way to get any. A destination in your account that doesn’t exist is also `Unroutable` .

> > > > > > Unknown

> > > > > SageMaker AI couldn’t check the destination, for example because you aren’t allowed to describe it or the check timed out. The destination might still be serving.

> > > > > Possible values:

> > > > > * `Routable`
> > > > > * `Unroutable`
> > > > > * `Unknown`

> > AsyncInferenceConfig -> (structure)

> > > The asynchronous inference settings of the routing target. Specify this setting to make the routing target asynchronous. If you omit it, the routing target serves real-time requests.

> > > Asynchronous routing targets aren’t available in all Amazon Web Services Regions. For information about asynchronous routing targets and Region availability, see [Send asynchronous requests through an inference router](https://docs.aws.amazon.com/sagemaker/latest/dg/inference-router-async.html) .

> > > Because `UpdateInferenceRouter` replaces all routing targets, include this setting in every update of an asynchronous routing target.

> > > For information about asynchronous routing targets, see [Send asynchronous requests through an inference router](https://docs.aws.amazon.com/sagemaker/latest/dg/inference-router-async.html) .

> > > OutputConfig -> (structure) [required]

> > > > The Amazon S3 locations for the results of asynchronous requests, and the notification settings.

> > > > S3OutputPath -> (string) [required]

> > > > > The Amazon S3 location for the results of successful requests. The bucket must be in the inference router’s Amazon Web Services Region and owned by the router’s account.

> > > > > Constraints:

> > > > > * min: `1`
> > > > > * max: `512`
> > > > > * pattern: `(https|s3)://([^/])/?(.*)`

> > > > S3FailurePath -> (string) [required]

> > > > > The Amazon S3 location for the failure details of failed requests. The bucket must be in the inference router’s Amazon Web Services Region and owned by the router’s account.

> > > > > Constraints:

> > > > > * min: `1`
> > > > > * max: `512`
> > > > > * pattern: `(https|s3)://([^/])/?(.*)`

> > > > KmsKeyId -> (string)

> > > > > The KMS key that encrypts the results in Amazon S3. You can specify a key ID, key ARN, alias name, or alias ARN. If any destination is in a different account or Amazon Web Services Region from the inference router, specify a key ARN or an alias ARN.

> > > > > Constraints:

> > > > > * min: `0`
> > > > > * max: `2048`
> > > > > * pattern: `[a-zA-Z0-9:/_-]*`

> > > > NotificationConfig -> (structure)

> > > > > The Amazon SNS topics that the inference router publishes to when a request completes or fails.

> > > > > SuccessTopic -> (string)

> > > > > > The ARN of the Amazon SNS topic to publish to when a request completes. The topic must be a standard topic, and can be in any Amazon Web Services Region.

> > > > > > Constraints:

> > > > > > * min: `20`
> > > > > > * max: `2048`
> > > > > > * pattern: `arn:aws[a-z-]*:sns:[a-z0-9-]*:[0-9]{12}:[a-zA-Z0-9_.-]+`

> > > > > ErrorTopic -> (string)

> > > > > > The ARN of the Amazon SNS topic to publish to when a request fails, or when the router can’t place it on any destination. The topic must be a standard topic, and can be in any Amazon Web Services Region.

> > > > > > Constraints:

> > > > > > * min: `20`
> > > > > > * max: `2048`
> > > > > > * pattern: `arn:aws[a-z-]*:sns:[a-z0-9-]*:[0-9]{12}:[a-zA-Z0-9_.-]+`

> > > ClientConfig -> (structure)

> > > > How the inference router submits requests to destinations.

> > > > MaxTimeInBacklogSeconds -> (integer)

> > > > > The maximum time, in seconds, that a request can wait at a destination before the inference router moves it to another destination. The default is 900 seconds.

> > > > > Constraints:

> > > > > * min: `60`
> > > > > * max: `21600`

> > RoutingPreference -> (string)

> > > How the inference router chooses among the routing target’s destinations. The default is `LocalRegionFirst` .

> > > > LocalRegionFirst

> > > The router uses destinations in its own Region first. When the request leaves the Region, the router spreads it across healthy destinations in other Regions.

> > > > InstancePoolOrder

> > > The router prefers destinations by instance type first and by Region second. Every destination must be an endpoint that lists the same two or more instance types, each with a different priority, in the same priority order. This value is supported only for asynchronous routing targets.

> > > With either value, the router uses destinations whose health or capacity is reduced after the others, but doesn’t exclude them.

> > > Possible values:

> > > * `LocalRegionFirst`
> > > * `InstancePoolOrder`

JSON Syntax:

```json
[
  {
    "Name": "string",
    "Destinations": [
      {
        "ResourceArn": "string",
        "RoutingStatus": "Routable"|"Unroutable"|"Unknown"
      }
      ...
    ],
    "AsyncInferenceConfig": {
      "OutputConfig": {
        "S3OutputPath": "string",
        "S3FailurePath": "string",
        "KmsKeyId": "string",
        "NotificationConfig": {
          "SuccessTopic": "string",
          "ErrorTopic": "string"
        }
      },
      "ClientConfig": {
        "MaxTimeInBacklogSeconds": integer
      }
    },
    "RoutingPreference": "LocalRegionFirst"|"InstancePoolOrder"
  }
  ...
]
```

`--client-token` (string)

> A unique, case-sensitive identifier that you provide to ensure the idempotency of the request. If you reuse a client token, use the same request parameters. Otherwise, the request fails.

> Constraints:

> * min: `1`
> * max: `64`
> * pattern: `[\x21-\x7E]+`

`--tags` (list)

> A list of key-value pairs to apply to the inference router. For information about tagging, see [Tagging Amazon Web Services resources](https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html) in the *Amazon Web Services General Reference* .

> Constraints:

> * min: `0`
> * max: `50`

> (structure)

> > A tag object that consists of a key and an optional value, used to manage metadata for SageMaker Amazon Web Services resources.

> > You can add tags to notebook instances, training jobs, hyperparameter tuning jobs, batch transform jobs, models, labeling jobs, work teams, endpoint configurations, and endpoints. For more information on adding tags to SageMaker resources, see [AddTags](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_AddTags.html) .

> > For more information on adding metadata to your Amazon Web Services resources with tagging, see [Tagging Amazon Web Services resources](https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html) . For advice on best practices for managing Amazon Web Services resources with tagging, see [Tagging Best Practices: Implement an Effective Amazon Web Services Resource Tagging Strategy](https://d1.awsstatic.com/whitepapers/aws-tagging-best-practices.pdf) .

> > Key -> (string) [required]

> > > The tag key. Tag keys must be unique per resource.

> > > Constraints:

> > > * min: `1`
> > > * max: `128`
> > > * pattern: `([\p{L}\p{Z}\p{N}_.:/=+\-@]*)`

> > Value -> (string) [required]

> > > The tag value.

> > > Constraints:

> > > * min: `0`
> > > * max: `256`
> > > * pattern: `([\p{L}\p{Z}\p{N}_.:/=+\-@]*)`

Shorthand Syntax:

```
Key=string,Value=string ...
```

JSON Syntax:

```json
[
  {
    "Key": "string",
    "Value": "string"
  }
  ...
]
```

`--cli-input-json` | `--cli-input-yaml` (string)
Reads arguments from the JSON string provided. The JSON string follows the format provided by `--generate-cli-skeleton`. If other arguments are provided on the command line, those values will override the JSON-provided values. It is not possible to pass arbitrary binary values using a JSON-provided value as the string will be taken literally. This may not be specified along with `--cli-input-yaml`.

`--generate-cli-skeleton` (string)
Prints a JSON skeleton to standard output without sending an API request. If provided with no value or the value `input`, prints a sample input JSON that can be used as an argument for `--cli-input-json`. Similarly, if provided `yaml-input` it will print a sample input YAML that can be used with `--cli-input-yaml`. If provided with the value `output`, it validates the command inputs and returns a sample output JSON for that command. The generated JSON skeleton is not stable between versions of the AWS CLI and there are no backwards compatibility guarantees in the JSON skeleton generated.

## Global Options

`--debug` (boolean)

Turn on debug logging.

`--endpoint-url` (string)

Override command’s default URL with the given URL.

`--no-verify-ssl` (boolean)

By default, the AWS CLI uses SSL when communicating with AWS services. For each SSL connection, the AWS CLI will verify SSL certificates. This option overrides the default behavior of verifying SSL certificates.

`--no-paginate` (boolean)

Disable automatic pagination. If automatic pagination is disabled, the AWS CLI will only make one call, for the first page of results.

`--output` (string)

The formatting style for command output.

* json
* text
* table
* yaml
* yaml-stream
* off

`--query` (string)

A JMESPath query to use in filtering the response data.

`--profile` (string)

Use a specific profile from your credential file.

`--region` (string)

The region to use. Overrides config/env settings.

`--version` (string)

Display the version of this tool.

`--color` (string)

Turn on/off color output.

* on
* off
* auto

`--no-sign-request` (boolean)

Do not sign requests. Credentials will not be loaded if this argument is provided.

`--ca-bundle` (string)

The CA certificate bundle to use when verifying SSL certificates. Overrides config/env settings.

`--cli-read-timeout` (int)

The maximum socket read time in seconds. If the value is set to 0, the socket read will be blocking and not timeout. The default value is 60 seconds.

`--cli-connect-timeout` (int)

The maximum socket connect time in seconds. If the value is set to 0, the socket connect will be blocking and not timeout. The default value is 60 seconds.

`--cli-binary-format` (string)

The formatting style to be used for binary blobs. The default format is base64. The base64 format expects binary blobs to be provided as a base64 encoded string. The raw-in-base64-out format preserves compatibility with AWS CLI V1 behavior and binary values must be passed literally. When providing contents from a file that map to a binary blob `fileb://` will always be treated as binary and use the file contents directly regardless of the `cli-binary-format` setting. When using `file://` the file contents will need to properly formatted for the configured `cli-binary-format`.

* base64
* raw-in-base64-out

`--no-cli-pager` (boolean)

Disable cli pager for output.

`--cli-auto-prompt` (boolean)

Automatically prompt for CLI input parameters.

`--no-cli-auto-prompt` (boolean)

Disable automatically prompt for CLI input parameters.

`--cli-error-format` (string)

The formatting style for error output. By default, errors are displayed in enhanced format.

* legacy
* json
* yaml
* text
* table
* enhanced

`--help` (boolean)

Display help for the command/subcommand.

## Output

InferenceRouterArn -> (string)

> The Amazon Resource Name (ARN) of the inference router.

> Constraints:

> * min: `20`
> * max: `2048`
> * pattern: `arn:aws[a-z-]*:sagemaker:[a-z0-9-]+:[0-9]{12}:inference-router/[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}`
