Request Syntax URI Request Parameters Request Body Response Syntax Response Elements Errors See Also

CreateExperiment

Creates an Evidently experiment. Before you create an experiment, you must create the feature to use for the experiment.

An experiment helps you make feature design decisions based on evidence and data. An experiment can test as many as five variations at once. Evidently collects experiment data and analyzes it by statistical methods, and provides clear recommendations about which variations perform better.

You can optionally specify a segment to have the experiment consider only certain audience types in the experiment, such as using only user sessions from a certain location or who use a certain internet browser.

Don't use this operation to update an existing experiment. Instead, use UpdateExperiment.

Request Syntax


POST /projects/project/experiments HTTP/1.1
Content-type: application/json

{
   "description": "string",
   "metricGoals": [ 
      { 
         "desiredChange": "string",
         "metricDefinition": { 
            "entityIdKey": "string",
            "eventPattern": "string",
            "name": "string",
            "unitLabel": "string",
            "valueKey": "string"
         }
      }
   ],
   "name": "string",
   "onlineAbConfig": { 
      "controlTreatmentName": "string",
      "treatmentWeights": { 
         "string" : number 
      }
   },
   "randomizationSalt": "string",
   "samplingRate": number,
   "segment": "string",
   "tags": { 
      "string" : "string" 
   },
   "treatments": [ 
      { 
         "description": "string",
         "feature": "string",
         "name": "string",
         "variation": "string"
      }
   ]
}

URI Request Parameters

The request uses the following URI parameters.

project

The name or ARN of the project that you want to create the new experiment in.

Length Constraints: Minimum length of 0. Maximum length of 2048.

Pattern: (^[a-zA-Z0-9._-]*$)|(arn:[^:]*:[^:]*:[^:]*:[^:]*:project/[a-zA-Z0-9._-]*)

Required: Yes

Request Body

The request accepts the following data in JSON format.

description

An optional description of the experiment.

Type: String

Length Constraints: Minimum length of 0. Maximum length of 160.

Pattern: .*

Required: No

metricGoals

An array of structures that defines the metrics used for the experiment, and whether a higher or lower value for each metric is the goal.

Type: Array of MetricGoalConfig objects

Array Members: Minimum number of 1 item. Maximum number of 3 items.

Required: Yes

name

A name for the new experiment.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 127.

Pattern: ^[-a-zA-Z0-9._]*$

Required: Yes

onlineAbConfig

A structure that contains the configuration of which variation to use as the "control" version. The "control" version is used for comparison with other variations. This structure also specifies how much experiment traffic is allocated to each variation.

This parameter is required.

Type: OnlineAbConfig object

Required: No

randomizationSalt

When Evidently assigns a particular user session to an experiment, it must use a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt. If you omit randomizationSalt, Evidently uses the experiment name as the randomizationSalt.

Type: String

Length Constraints: Minimum length of 0. Maximum length of 127.

Pattern: .*

Required: No

samplingRate

The portion of the available audience that you want to allocate to this experiment, in thousandths of a percent. The available audience is the total audience minus the audience that you have allocated to overrides or current launches of this feature.

This is represented in thousandths of a percent. For example, specify 10,000 to allocate 10% of the available audience.

Type: Long

Valid Range: Minimum value of 0. Maximum value of 100000.

Required: No

segment

Specifies an audience segment to use in the experiment. When a segment is used in an experiment, only user sessions that match the segment pattern are used in the experiment.

Type: String

Length Constraints: Minimum length of 0. Maximum length of 2048.

Pattern: (^[-a-zA-Z0-9._]*$)|(arn:[^:]*:[^:]*:[^:]*:[^:]*:segment/[-a-zA-Z0-9._]*)

Required: No

tags

Assigns one or more tags (key-value pairs) to the experiment.

Tags can help you organize and categorize your resources. You can also use them to scope user permissions by granting a user permission to access or change only resources with certain tag values.

Tags don't have any semantic meaning to AWS and are interpreted strictly as strings of characters.

You can associate as many as 50 tags with an experiment.

For more information, see Tagging AWS resources.

Type: String to string map

Key Length Constraints: Minimum length of 1. Maximum length of 128.

Key Pattern: ^(?!aws:)[a-zA-Z+-=._:/]+$

Value Length Constraints: Minimum length of 0. Maximum length of 256.

Required: No

treatments

An array of structures that describe the configuration of each feature variation used in the experiment.

Type: Array of TreatmentConfig objects

Array Members: Minimum number of 0 items. Maximum number of 5 items.

Required: Yes

Response Syntax


HTTP/1.1 200
Content-type: application/json

{
   "experiment": { 
      "arn": "string",
      "createdTime": number,
      "description": "string",
      "execution": { 
         "endedTime": number,
         "startedTime": number
      },
      "lastUpdatedTime": number,
      "metricGoals": [ 
         { 
            "desiredChange": "string",
            "metricDefinition": { 
               "entityIdKey": "string",
               "eventPattern": "string",
               "name": "string",
               "unitLabel": "string",
               "valueKey": "string"
            }
         }
      ],
      "name": "string",
      "onlineAbDefinition": { 
         "controlTreatmentName": "string",
         "treatmentWeights": { 
            "string" : number 
         }
      },
      "project": "string",
      "randomizationSalt": "string",
      "samplingRate": number,
      "schedule": { 
         "analysisCompleteTime": number
      },
      "segment": "string",
      "status": "string",
      "statusReason": "string",
      "tags": { 
         "string" : "string" 
      },
      "treatments": [ 
         { 
            "description": "string",
            "featureVariations": { 
               "string" : "string" 
            },
            "name": "string"
         }
      ],
      "type": "string"
   }
}

Response Elements

If the action is successful, the service sends back an HTTP 200 response.

The following data is returned in JSON format by the service.

experiment

A structure containing the configuration details of the experiment that you created.

Type: Experiment object

Errors

For information about the errors that are common to all actions, see Common Errors.

AccessDeniedException

You do not have sufficient permissions to perform this action.

HTTP Status Code: 403

ConflictException

A resource was in an inconsistent state during an update or a deletion.

HTTP Status Code: 409

ResourceNotFoundException

The request references a resource that does not exist.

HTTP Status Code: 404

ServiceQuotaExceededException

The request would cause a service quota to be exceeded.

HTTP Status Code: 402

ValidationException

The value of a parameter in the request caused an error.

HTTP Status Code: 400