Create a model card

Important

Custom IAM policies that allow Amazon SageMaker Studio or Amazon SageMaker Studio Classic to create Amazon SageMaker resources must also grant permissions to add tags to those resources. The permission to add tags to resources is required because Studio and Studio Classic automatically tag any resources they create. If an IAM policy allows Studio and Studio Classic to create resources but does not allow tagging, "AccessDenied" errors can occur when trying to create resources. For more information, see Provide permissions for tagging SageMaker resources.

AWS managed policies for Amazon SageMaker that give permissions to create SageMaker resources already include permissions to add tags while creating those resources.

You can create an Amazon SageMaker Model Card using either the SageMaker console or the SageMaker Python SDK. You can also use the API operations directly. For more information about the API operations, see Low-level SageMaker APIs for model cards.

Create a model card using the SageMaker console

Go to the Amazon SageMaker console. In the navigation pane, under Governance, choose Model cards. On the upper right-hand corner, choose Create model card.

Go through the four steps in the Create model card prompt to document details about your model.

Step 1: Enter model details and intended use

If your model is an AWS resource, specify the exact model name in this field to auto-populate model details. To browse existing model names, see Models in the Amazon SageMaker console. Each unique model name can have only one associated model card.

If your model is not an AWS resource, provide a unique name for your model. To add a model as an AWS resource, see Create a model in the Amazon SageMaker Developer Guide. Alternatively, you can add your model as a model package using SageMaker Marketplace or SageMaker Model Registry.

For more information on intended uses, see Intended uses of a model. For more information on risk ratings, see Risk ratings.

Step 2: Enter training details

Add any training details, training observations, datasets, hyperparameters, and details about the model's objective function to the model card.

The objective function in a model card can be any function that is optimized during training. This can include, but is not limited to, cost functions, loss functions, or objective metrics. In this section, document the objective function that is most critical to training your model.

We recommend that you catalog the following attributes of your objective function:

• Optimization direction

• Metric

• Description

For example, you might minimize (optimization direction) cross entropy loss (metric) for a binary classification problem (description) or maximize likelihood for logistic regression. Additionally, you can provide notes about why you chose this objective function over others.

Step 3: Enter evaluation details

If you have existing evaluation reports generated by SageMaker Clarify or Model Monitor, either provide an S3 URI for those reports or upload them manually to add them to the model card.

For more information on SageMaker Clarify, see Run SageMaker Clarify Processing Jobs for Bias Analysis and Explainability.

For more information on monitoring drift in model quality metrics using Model Monitor, see Monitor model quality.

To add your own evaluation report, choose Generic model card evaluation. All model card evaluation reports must be in the Model card JSON schema.

Step 4: Enter additional details

Add custom model card detail fields for any additional information that you want to address on your model card. For example, you might include the custom field Line of business with a value of Personal finance.

Save model card

After reviewing the information in your model card, choose Save in the lower right-hand corner to save your model card.

Create a model card using the SageMaker Python SDK

Before creating a model card, you must first define the content of your model card. When using the SageMaker Python SDK, model content consists of a model overview, training details, intended uses, evaluation details, and additional information.

You can create model cards for:

• Models that are hosted within SageMaker

• Model packages (models) within the SageMaker Model Registry

• Models that are hosted or registered outside of SageMaker

You can also create model cards without associating any models to them.

We recommend adding the models that you've trained to the SageMaker Model Registry. The model registry helps you catalog models and track model versions. When you create a model card, the information about the model from the model registry automatically populates the model card. You can edit the model card or add information to it after you create it.

For information about using the model registry, see Model Registration Deployment with Model Registry. For information about creating a model card from a model registry, see Create a model card for your model in the SageMaker Model Registry.

Note

To use model cards with the SageMaker Python SDK, you first need to establish a SageMaker Session. For more information, see Session in the SageMaker Python SDK API reference.

To create a model card for models that aren't in the SageMaker Model Registry, see Create a model that isn't in the model registry.

Create a model that isn't in the model registry

Use the information in the following sections to create a model card for a model that you haven't added to the model registry.

Step 1: Define model overview

Define an overview of your model.

model_overview = ModelOverview.from_model_name(
model_name=model_name,
sagemaker_session=sagemaker_session,
model_description="A-description-of-your-model",
problem_type="Problem-type", # For example, "Binary Classification"
algorithm_type="Algorithm-type", # For example, "Logistic Regression"
model_creator="Name-of-model-creator",
model_owner="Name-of-model-owner",
)

If your model is an AWS resource, then overview information such as the model ARN, inference container URI, and the S3 location of model artifacts is automatically retrievable. Print the associated AWS metadata with the following commands:

print(model_overview.model_id)
print(model_overview.inference_environment.container_image)
print(model_overview.model_artifact)

Step 2: Define training details

To define your model's training details, you must first define its objective function.

objective_function = ObjectiveFunction(
    function=Function(
        function=ObjectiveFunctionEnum.MINIMIZE,
        facet=FacetEnum.LOSS,
    ),
    notes="An-explanation-about-objective-function",
)

Next, you can define your training details using your existing model overview, session, and objective function. Add any training observations here.

training_details = TrainingDetails.from_model_overview(
    model_overview=model_overview,
    sagemaker_session=sagemaker_session,
    objective_function=objective_function,
    training_observations="Model-training-observations",
)

Once again, if your model is an AWS resource, certain training details are autopopulated. Print the training job ARN, training container URI, and training metrics with the following commands:

print(training_details.training_job_details.training_arn)
print(training_details.training_job_details.training_environment.container_image) 
print([{"name": i.name, "value": i.value} for i in training_details.training_job_details.training_metrics]) 

Define evaluation details

To define your model's evaluation details, you must first define one or more metric groups to describe metrics used for any evaluation jobs.

my_metric_group = MetricGroup(
name="binary classification metrics",
metric_data=[Metric(name="accuracy", type=MetricTypeEnum.NUMBER, value=0.5)]
)

Next, you can define your evaluation details using evaluation metrics and datasets for each evaluation job. Add any evaluation observations here and give your evaluation job a unique name.

evaluation_details = [
    EvaluationJob(
        name="Example-evaluation-job",
        evaluation_observation="Evaluation-observations",
        datasets=["s3://path/to/evaluation/data"],
        metric_groups=[my_metric_group],
    )
]

If you have existing evaluation reports generated by SageMaker Clarify or SageMaker Model Monitor, upload them to Amazon S3 and provide an S3 URI to automatically parse evaluation metrics. To add your own generic model card evaluation report, provide a report in the evaluation results JSON format.

report_type = "clarify_bias.json"
example_evaluation_job.add_metric_group_from_json(
    f"example_metrics/{report_type}", EvaluationMetricTypeEnum.CLARIFY_BIAS
) 

Step 3: Define intended uses

Define the intended uses of the model, including the general purpose of the model and the use cases for which it was intended. It is also recommended to include any factors that potentially a this model's efficacy in a particular use case and your organization's risk rating of the model. For more information, see Intended uses of a modeland Risk ratings.

intended_uses = IntendedUses(
purpose_of_model="Purpose-of-the-model",
intended_uses="The-intended-uses-of-this-model",
factors_affecting_model_efficiency="Any-factors-effecting-model-efficacy",
risk_rating=RiskRatingEnum.LOW,
explanations_for_risk_rating="Explanation-for-low-risk-rating",
)

Define additional information

Lastly, you can add additional custom information to your model card. You can document any ethical considerations, caveats, and recommendations about the model. You can also add any custom details that you would like in the form of key-value pairs.

additional_information = AdditionalInformation(
ethical_considerations="Any-ethical-considerations",
caveats_and_recommendations="Any-caveats-and-recommendations",
custom_details={"custom details1": "details-value"},
)

Step 4: Create model card

Name your model card, define a model card, and then use that definition to create a model card using the SageMaker Python SDK.

model_card_name = "my-model-card"
my_card = ModelCard(
    name=model_card_name,
    status=ModelCardStatusEnum.DRAFT,
    model_overview=model_overview,
    training_details=training_details,
    intended_uses=intended_uses,
    evaluation_details=evaluation_details,
    additional_information=additional_information,
    sagemaker_session=sagemaker_session,
)
my_card.create()

Create a model card for your model in the SageMaker Model Registry

Before you begin creating a model card, make sure that you've created a model package group and a model package. For more information about using model registry, see Model Registration Deployment with Model Registry.

Important

You must have permissions to use the operations in SageMaker Model Registry. We recommend using AmazonSageMakerModelRegistryFullAccess AWS managed policy. For more information about the managed policy, see AWS Managed Policies for Model Registry.

Use the SageMaker Python SDK to create a model card for a model package within the SageMaker Model Registry. A model package is a model that you've trained. When you create a model card, Amazon SageMaker Model Cards automatically imports the data from the model package into the model card.

When you create a model card for a model package, Amazon SageMaker Model Card uses the DescribeModelPackage operation to add the data from the model package to the model card. The following are examples of the fields that can be imported from a model package into a model card:

• ModelDataUrl

• ModelPackageDescription

• ModelPackageGroupName

• ModelPackageStatus

• ModelPackageVersion

Use the following code to define the model package and create a model card from it:

mp_details = ModelPackage.from_model_package_arn(
    model_package_arn="example_model_package_arn",
    sagemaker_session=sagemaker_session,
)

model_card_name = "example-model-card"
my_card = ModelCard(
    name=model_card_name,
    status=ModelCardStatusEnum.status,
    model_package_details=mp_details,
    sagemaker_session=sagemaker_session,
)
my_card.create()
            

For status, you're specifying the approval status of the model card. If you don't specify a status, SageMaker Model Cards uses the default value of DRAFT. If you don't specify a SageMaker session, SageMaker Model Cards uses the default SageMaker session.

You must specify a name for the model and the Amazon Resource Name (ARN) of the model package. For information about getting the Amazon Resource Name (ARN) for the model package, see View and Update the Details of a Model Version (Boto3).

The model card that you've created from the model package might have information that is either missing or inaccurate. You can add information to the model card or edit it. For more information about managing your model cards, see Model cards actions.

SageMaker Model Registry supports versioning of your model packages. You can version your model package and create a model card for each version. The information from model cards of preceding versions carry over to model cards created from subsequent versions. For example, you could have version 1, version 2, and version 3 of a model package. Suppose you've already created a model card for version 1, but you haven't created one for version 2. If you create a model card for version 3, Amazon SageMaker Model Cards automatically carries over the information from the model card for version 1 to the model card for version 3.

Note

You can also create model cards for model packages that don't use versioning. However, most machine learning workflows involve multiple versions of the same model, so we recommend doing the following:

1. Creating a version for each model package

2. Creating a model card for each version of the model package