Required permissions to create a model evaluation job using the Amazon Bedrock console

When you create a model evaluation job using the Amazon Bedrock console, you must specify the correct CORS permissions in your Amazon S3 bucket. To learn more, see Required Cross Origin Resource Sharing (CORS) permission on S3 buckets.

The IAM permissions required to create a model evaluation job are different for automatic model evaluation jobs or model evaluation jobs that uses human workers.

Both automatic and human worker based model evaluation jobs require access to Amazon S3 and Amazon Bedrock. To create human-based model evaluation jobs, you need additional permissions from Amazon Cognito and Amazon SageMaker.

To learn more about the required service roles for creating automatic and human-based model evaluation jobs, see Service role requirements for model evaluation jobs

Required console permissions to create an automatic model evaluation job

The following policy contains the minimum set of IAM actions and resources in Amazon Bedrock and Amazon S3 that are required to create an automatic model evaluation job using the Amazon Bedrock console.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "BedrockConsole",
            "Effect": "Allow",
            "Action": [
               "bedrock:CreateEvaluationJob",
               "bedrock:GetEvaluationJob",
               "bedrock:ListEvaluationJobs",
               "bedrock:StopEvaluationJob",
               "bedrock:GetCustomModel",
               "bedrock:ListCustomModels",
               "bedrock:CreateProvisionedModelThroughput",
               "bedrock:UpdateProvisionedModelThroughput",
               "bedrock:GetProvisionedModelThroughput",
               "bedrock:ListProvisionedModelThroughputs",
               "bedrock:ListTagsForResource",
               "bedrock:UntagResource",
               "bedrock:TagResource"
            ],
            "Resource": "*"
        },
        {
            "Sid": "AllowConsoleS3AccessForModelEvaluation",
            "Effect": "Allow",
            "Action": [
              "s3:GetObject",
              "s3:GetBucketCORS",
              "s3:ListBucket",
              "s3:ListBucketVersions",
              "s3:GetBucketLocation"
            ],
            "Resource": "*"
        }
    ]
} 

Required permissions to create a human-based model evaluation job

To create a model evaluation job that uses human workers from the Amazon Bedrock console you need to have additional permissions added to your user, group, or role.

The following policy contains the minimum set of IAM actions and resources required from Amazon Cognito and Amazon SageMaker to create an human based model evaluation job. You must append this policy to the base policy requirements for an automatic model evaluation jobs.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowCognitionActionsForWorkTeamCreations",
            "Effect": "Allow",
            "Action": [
               "cognito-idp:CreateUserPool",
               "cognito-idp:CreateUserPoolClient",
               "cognito-idp:CreateGroup",
               "cognito-idp:AdminCreateUser",
               "cognito-idp:AdminAddUserToGroup",
               "cognito-idp:CreateUserPoolDomain",
               "cognito-idp:UpdateUserPool",
               "cognito-idp:ListUsersInGroup",
               "cognito-idp:ListUsers",
               "cognito-idp:AdminRemoveUserFromGroup"
            ],
            "Resource": "*"
        },
        {
            "Sid": "AllowSageMakerResourceCreation",
            "Effect": "Allow",
            "Action": [
                "sagemaker:CreateFlowDefinition",
                "sagemaker:CreateWorkforce",
                "sagemaker:CreateWorkteam",
                "sagemaker:DescribeFlowDefinition",
                "sagemaker:DescribeHumanLoop",
                "sagemaker:ListFlowDefinitions",
                "sagemaker:ListHumanLoops",
                "sagemaker:DescribeWorkforce",
                "sagemaker:DescribeWorkteam",
                "sagemaker:ListWorkteams",
                "sagemaker:ListWorkforces",
                "sagemaker:DeleteFlowDefinition",
                "sagemaker:DeleteHumanLoop",
                "sagemaker:RenderUiTemplate",
                "sagemaker:StartHumanLoop",
                "sagemaker:StopHumanLoop"
            ],
            "Resource": "*"
        }
    ]
} 

Required Cross Origin Resource Sharing (CORS) permission on S3 buckets

When you create a model evaluation job that uses the Amazon Bedrock console, you must specify a CORS configuration on the S3 bucket.

A CORS configuration is a document that defines rules that identify the origins that you will allow to access your bucket, the operations (HTTP methods) supported for each origin, and other operation-specific information. To learn more about setting the required CORS configuration using the S3 console, see Configuring cross-origin resource sharing (CORS) in the Amazon S3 User Guide.

The following is the minimal required CORS configuration for S3 buckets.

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET",
            "PUT",
            "POST",
            "DELETE"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": ["Access-Control-Allow-Origin"]
    }
]