Prerequisites for batch inference

To perform batch inference, you must fulfill the following prerequisites:

1. Ensure that an IAM identity has the necessary permissions to submit and manage batch inference jobs.

2. Prepare your dataset and upload it to an Amazon S3 bucket.

3. Create an S3 bucket for your output data.

The following steps are optional:

• Create a custom AWS Identity and Access Management (IAM) service role for your batch inference job with the proper permissions. You can skip this prerequisite if you plan to use the AWS Management Console to automatically create a service role for you.

• Set up a VPC to protect the data in your S3 while carrying out batch inference. You can skip this step if you don't need to use a VPC.

Required permissions for batch inference

For an IAM identity to submit and manage batch inference jobs, you must configure it with the necessary permissions. You can attach the AmazonBedrockFullAccess policy to grant the proper permissions to the role.

To restrict permissions to only actions that are used for batch inference, attach the following identity-based policy to an IAM role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "BatchInference",
            "Effect": "Allow",
            "Action": [  
                "bedrock:ListFoundationModels",
                "bedrock:GetFoundationModel",
                "bedrock:TagResource", 
                "bedrock:UntagResource", 
                "bedrock:ListTagsForResource",
                "bedrock:CreateModelInvocationJob",
                "bedrock:GetModelInvocationJob",
                "bedrock:ListModelInvocationJobs",
                "bedrock:StopModelInvocationJob"
            ],
            "Resource": "*"
        }
    ]   
}

To further restrict permissions, you can omit actions, or you can specify resources and condition keys by which to filter permissions. For more information about actions, resources, and condition keys, see the following topics in the Service Authorization Reference:

• Actions defined by Amazon Bedrock – Learn about actions, the resource types that you can scope them to in the Resource field, and the condition keys that you can filter permissions on in the Condition field.

• Resource types defined by Amazon Bedrock – Learn about the resource types in Amazon Bedrock.

• Condition keys for Amazon Bedrock – Learn about the condition keys in Amazon Bedrock.

Batch inference jobs use the foundation-model, custom-model, and model-invocation-job resource types. You can scope down permissions by specifying these resources in the Resource field. For example, the following policy allows a user with the account ID 123456789012 to create batch inference jobs in the us-west-2 region, using the Anthropic Claude 3 Haiku model:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CreateBatchInferenceJob",
            "Effect": "Allow",
            "Action": [
                "bedrock:CreateModelInvocationJob"
            ],
            "Resource": [
                "arn:aws:bedrock:us-west-2::foundation-model/anthropic.claude-3-haiku-20240307-v1:0"
                "arn:aws:bedrock:us-west-2:123456789012:model-invocation-job/*"
            ]
        }
    ]
}

Topics

• Format and upload your batch inference data
• Protect batch inference jobs using a VPC