Create a SageMaker Experiment - Amazon SageMaker

Create a SageMaker Experiment

Create a Amazon SageMaker experiment to track your SageMaker training, processing, and transform jobs.

The following procedure shows you how to create a SageMaker experiment for a SageMaker training, processing, or transform job. Steps labeled as (Studio) describe how to view the experiment in Amazon SageMaker Studio. You don't have to run the experiment in Studio to view the experiment in Studio.

For a tutorial that shows this functionality in an existing SageMaker Studio notebook, see Track and Compare Tutorial.

  1. Import the sys module to install the SDKs.

    import sys
  2. (Optional) The SageMaker Python SDK, comes preinstalled in SageMaker Studio. If you plan to run your code outside Studio, install the SageMaker Python SDK.

    !{sys.executable} -m pip install sagemaker
  3. Install the SageMaker Experiments Python SDK.

    !{sys.executable} -m pip install sagemaker-experiments
  4. Import modules.

    import time from time import strftime import sagemaker from smexperiments.experiment import Experiment from smexperiments.trial import Trial from smexperiments.trial_component import TrialComponent from smexperiments.tracker import Tracker
  5. Get the execution role and create the SageMaker session.

    role = sagemaker.get_execution_role() sm_sess = sagemaker.session.Session()
  6. Create a SageMaker experiment. The experiment name must be unique in your account.

    Note

    The tags parameter is optional. You can search for the tag using Studio, the SageMaker console, and the SDK. Tags can also be applied to trials and trial components. For information on how to search tags using Studio, see Search by Tag.

    create_date = strftime("%Y-%m-%d-%H-%M-%S") demo_experiment = Experiment.create(experiment_name = "DEMO-{}".format(create_date), description = "Demo experiment", tags = [{'Key': 'my-experiments', 'Value': 'demo1'}])
  7. (Studio) To view the experiment in SageMaker Studio, in the left sidebar, choose the SageMaker Experiment List icon ( ) to display the experiments browser.

    After the code runs, the experiment list contains the new experiment. It might take a moment for the list to refresh and display the experiment. The filter on the experiment tag is also displayed. Only experiments that have a matching tag are displayed. Your list should look similar to the following:

  8. Create a trial for the experiment. The trial name must be unique in your account.

    demo_trial = Trial.create(trial_name = "DEMO-{}".format(create_date), experiment_name = demo_experiment.experiment_name, tags = [{'Key': 'my-experiments', 'Value': 'demo1'}])
  9. (Studio) In the experiment list, double-click the experiment to display a list of the trials in the experiment (this example has one trial). Your list should look similar to the following:

  10. Create a trial component as part of the trial. The trial component is the SageMaker job.

    Add the ExperimentConfig parameter to the appropriate method. The SageMaker jobs listed in the following table are supported.

    Job SageMaker Python SDK method Boto3 method
    Training Estimator.fit CreateTrainingJob
    Processing Processor.run CreateProcessingJob
    Transform Transformer.transform CreateTransformJob

    The following examples are for a training job. The Tags parameter adds a tag to the trial component. ExperimentName isn't specified because the trial was associated with the experiment when the trial was created in an earlier step.

    Using the SageMaker Python SDK

    sagemaker.estimator.Estimator( ..., sagemaker_session = sm_sess, tags = [{'Key': 'my-experiments', 'Value': 'demo2'}]) estimator.fit( ..., experiment_config = { # "ExperimentName" "TrialName" : demo_trial.trial_name, "TrialComponentDisplayName" : "TrainingJob", })

    Using Boto3

    create_training_job( ..., "ExperimentConfig": { # "ExperimentName" "TrialName" : demo_trial.trial_name, "TrialComponentDisplayName" : "TrainingJob", }, "Tags": [{'Key': 'my-experiments', 'Value': 'demo2'}])
  11. (Studio) In the trial list, double-click the trial to display a list of the components in the trial (this example has one trial). Your list should look similar to the following:

  12. (Studio) To view information about the experiment, trial, and job (trial component), see View and Compare SageMaker Experiments, Trials, and Trial Components.

To clean up the resources you created, see Clean Up SageMaker Experiment Resources.