SageMakerGatewayRuntime / Client / exceptions / BadGatewayException
BadGatewayException¶
- class SageMakerGatewayRuntime.Client.exceptions.BadGatewayException¶
502 – The gateway could not get a usable response from the inference gateway. The
Messageprefix says why:Global Inference Gateway: could not reach the inference gateway: <reason>– the request could not be sent, or the connection closed before a response arrived (for example, the connection failed, or the inference gateway closed it after reading the request).Global Inference Gateway: timed out waiting for the inference gateway– on the buffered operation, the inference gateway stopped sending partway through the response body and the gateway’s read timeout expired. (The same text on a GatewayTimeoutException means no response arrived.)Any other
Global Inference Gateway:message – the gateway ended a response before it was complete, for example because it exceeded the 10 MiB limit of the buffered operation, the request exceeded its maximum duration, or the inference gateway was taken out of service.inference gateway: <status> <message>– the inference gateway serving the model returned a 502, 500, 503 or another 5xx error. Its status code is in the message (the response status is 502), and up to 256 bytes of its error message are quoted.inference gateway: response ended unexpectedly– the inference gateway’s connection failed while it was sending the response.
The inference gateway names a component in the prefix when it reports one:
inference gateway (<component>):, where the component isconfig(the inference gateway’s configuration and proxy layer),scheduler(its request scheduler),model selector(its model selector) ormodel server(the model server).Not modeled as retryable: a 502 can mean the inference already ran, and the gateway may already have tried another inference gateway. Standard SDK retry modes may still retry a 5xx status, which would run (and bill) the inference again; set max attempts to 1 if a repeated inference is unacceptable.
Example
try: ... except client.exceptions.BadGatewayException as e: print(e.response)
- response¶
The parsed error response. All exceptions have a top level
Errorkey that provides normalized access to common exception atrributes. All other keys are specific to this service or exception class.Syntax
{ 'Message': 'string', 'Error': { 'Code': 'string', 'Message': 'string' } }
Structure
(dict) –
502 – The gateway could not get a usable response from the inference gateway. The
Messageprefix says why:Global Inference Gateway: could not reach the inference gateway: <reason>– the request could not be sent, or the connection closed before a response arrived (for example, the connection failed, or the inference gateway closed it after reading the request).Global Inference Gateway: timed out waiting for the inference gateway– on the buffered operation, the inference gateway stopped sending partway through the response body and the gateway’s read timeout expired. (The same text on a GatewayTimeoutException means no response arrived.)Any other
Global Inference Gateway:message – the gateway ended a response before it was complete, for example because it exceeded the 10 MiB limit of the buffered operation, the request exceeded its maximum duration, or the inference gateway was taken out of service.inference gateway: <status> <message>– the inference gateway serving the model returned a 502, 500, 503 or another 5xx error. Its status code is in the message (the response status is 502), and up to 256 bytes of its error message are quoted.inference gateway: response ended unexpectedly– the inference gateway’s connection failed while it was sending the response.
The inference gateway names a component in the prefix when it reports one:
inference gateway (<component>):, where the component isconfig(the inference gateway’s configuration and proxy layer),scheduler(its request scheduler),model selector(its model selector) ormodel server(the model server).Not modeled as retryable: a 502 can mean the inference already ran, and the gateway may already have tried another inference gateway. Standard SDK retry modes may still retry a 5xx status, which would run (and bill) the inference again; set max attempts to 1 if a repeated inference is unacceptable.
Message (string) –
A description of the error. It starts with a prefix such as
Global Inference Gateway:orinference gateway:that says where the error came from.Error (dict) – Normalized access to common exception attributes.
Code (string) – An identifier specifying the exception type.
Message (string) – A descriptive message explaining why the exception occured.