The following topics show how to get information about and debug your lifecycle configurations.
Verify lifecycle configuration process from
CloudWatch Logs
Lifecycle configurations only log STDOUT
and STDERR
.
STDOUT
is the default output for bash scripts. You can write to
STDERR
by appending >&2
to the end of a bash command.
For example, echo 'hello'>&2
.
Logs for your lifecycle configurations are published to your AWS account using
Amazon CloudWatch. These logs can be found in the /aws/sagemaker/studio
log stream
in the CloudWatch console.
Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/
. -
Choose Logs from the left navigation pane. From the dropdown menu, select Log groups.
-
On the Log groups page, search for
aws/sagemaker/studio
. -
Select the log group.
-
On the Log group details page, choose the Log streams tab.
-
To find the logs for a specific app, search the log streams using the following format:
domain-id
/user-profile-name
/app-type
/app-name
The following search string finds the lifecycle configuration logs for the domain
d-m85lcu8vbqmz
, user profilei-sonic-js
, application typeJupyterLab
, and application nametest-lcc-echo
:d-m85lcu8vbqmz/i-sonic-js/JupyterLab/test-lcc-echo
-
To view the script execution logs, select the log stream appended with
LifecycleConfigOnStart
.
Lifecycle configuration timeout
There is a lifecycle configuration timeout limitation of 5 minutes. If a lifecycle configuration script takes longer than 5 minutes to run, you get an error.
To resolve this error, make sure that your lifecycle configuration script completes in less than 5 minutes.
To help decrease the runtime of scripts, try the following:
-
Reduce unnecessary steps. For example, limit which conda environments to install large packages in.
-
Run tasks in parallel processes.
-
Use the nohup command in your script to make sure that hangup signals are ignored so that the script runs without stopping.