SageMakerGatewayRuntime

Client

class SageMakerGatewayRuntime.Client

A low-level client representing Amazon SageMaker Gateway Runtime

Use the Amazon SageMaker Gateway Runtime API to run inference through a Global Inference Gateway. A Global Inference Gateway routes each request to an inference gateway that serves the model named in the request.

The request body is an OpenAI-compatible JSON object, sent with Content-Type: application/json, and the model is selected by the model key in that body. The content of the body is passed to the model server unchanged, except for the stream flag, which each operation sets.

Requests are signed with Signature Version 4 under the sagemaker signing name. IAM actions are in the sagemaker namespace, for example sagemaker:InvokeGlobalInferenceGateway. The default regional endpoint is gateway-runtime.sagemaker.{region}.amazonaws.com.

client = session.create_client('sagemaker-gateway-runtime')

These are the available methods:

Client Exceptions

Client exceptions are available on a client instance via the exceptions property. For more detailed instructions and examples on the exact usage of client exceptions, see the error handling user guide.

The available client exceptions are: