Format and upload your batch inference data

To prepare inputs for batch inference, create a .jsonl file in the following format:

{ "recordId" : "11 character alphanumeric string", "modelInput" : {JSON body} }
...

Each line contains a JSON object with a recordId field and a modelInput field containing the request body for an input you want to submit. The format of the modelInput JSON object must match the body field for the model that you use in the InvokeModel request. For more information, see Inference request parameters and response fields for foundation models.

Note

If you omit the recordId field, Amazon Bedrock adds it in the output.

For example, you might provide a JSONL file containing the following line if you plan to run batch inference using the Anthropic Claude 3 Haiku model:

{
    "recordId": "CALL0000001", 
    "modelInput": {
        "anthropic_version": "bedrock-2023-05-31", 
        "max_tokens": 1024,
        "messages": [ 
            { 
                "role": "user", 
                "content": [
                    {
                        "type": "text", 
                        "text": "Summarize the following call transcript: ..." 
                    } 
                ]
            }
        ]
    }
}

After preparing your input files, upload them to an S3 bucket. Attach the following permissions to your batch inference service role and replace ${{s3-bucket-input}} with the bucket that you uploaded the input files to and ${{s3-bucket-output}} with the bucket that you want to write the output files to.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::${{s3-bucket-input}}", 
                "arn:aws:s3:::${{s3-bucket-input}}/*",
                "arn:aws:s3:::${{s3-bucket-output}}",
                "arn:aws:s3:::${{s3-bucket-output}}/*"
            ],
            "Effect": "Allow"
        }
    ]
}