Skip to content

Experiments API

Experiments

An experiment represents an A/B test. It has a goal and multiple groups with different treatments. Experiments can be limited duration or indefinite.

An experiment resource has following attributes:

Attribute Type Description
id UUID An unique ID for the experiment
is_draft boolean Whether the experiment is a draft or published. Once published, the experiment cannot be changed back to draft
created_at datetime Create time of the experiment
created_by_user object The user who created this experiment
created_by_user_id string ID of the user who created this experiment
updated_at datetime Update time of the experiment
updated_by_user object The user who last updated this experiment
updated_by_user_id string ID of the user who last updated this experiment
organization_id UUID ID of the organization that owns this experiment
organization object The organization that owns this experiment
goal_id UUID ID of the goal of this experiment
goal object The goal of this experiment
groups array of objects List of the experiment's groups
name string The name of the experiment
starts_at datetime The start time of the experiment
ends_at datetime The end time of the experiment
group_reset_interval string When to reset group's visitors, one of: full (default), 1w or 1d
rooms array of objects List of rooms where the experiment treatments can be triggered. If the list is empty, treatment can be triggered in any of the organization's rooms
room_ids array of UUIDs List of room IDs where the experiment treatments can be triggered. If the list is empty, treatment can be triggered in any of the organization's rooms

Experiment groups

Attribute Type Description
id UUID The unique identifier of this experiment group
created_at datetime Create time of the group
updated_at datetime Update time of the group
order integer The index of this group in the order of groups within the experiment
name string Name of this group
percentage integer The percentage this group has within the experiments group distribution. Percentages of groups within a experiment must add up to 100
group_type string Group type. One of control, user_defined, target_algorithm, target_random or null
treatment object Rule that is ran for visitors in this group

List experiments

GET /api/v5/orgs/<organization_id>/experiments

Retrieve details of an experiment

GET /api/v5/orgs/<organization_id>/experiments/<experiment_id>

Example of experiment details:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
{
    "is_draft": false,
    "created_at": "2019-02-13T11:31:36.042",
    "created_by_user": {
        "id": "97abf1ee-6308-11e7-9d09-00163ec04961",
        "first_name": "John",
        "last_name": "Doe",
        "full_name": "John Doe",
        "organization_id": "aabefa3e-1c8e-4690-99e4-2683c77d4ff0",
        "is_bot": false
    },
    "created_by_user_id": "97abf1ee-6308-11e7-9d09-00163ec04961",
    "goal_id": "0a534844-869a-11e9-8dcc-27f1dd2d9764",
    "goal": {
        "id": "0a534844-869a-11e9-8dcc-27f1dd2d9764",
        "name": "Reached checkout page",
        "organization": {
            "id": "aabefa3e-1c8e-4690-99e4-2683c77d4ff0",
            "name": "Company Inc."
        },
        "organization_id": "aabefa3e-1c8e-4690-99e4-2683c77d4ff0"
    },
    "groups": [
        {
            "name": "A",
            "percentage": 90,
            "group_type": "control",
            "treatment": {}
        },
        {
            "name": "B",
            "percentage": 10,
            "group_type": "user_defined",
            "treatment": {}
        }
    ],
    "group_reset_interval": "full",
    "name": "Hello experiment",
    "starts_at": "2019-05-08T13:02:35.123Z",
    "ends_at": "2019-05-22T13:02:35.123Z",
    "updated_at": "2019-05-07T11:00:00.000Z",
    "updated_by_user": {
        "id": "97abf1ee-6308-11e7-9d09-00163ec04961",
        "first_name": "John",
        "last_name": "Doe",
        "full_name": "John Doe",
        "organization_id": "aabefa3e-1c8e-4690-99e4-2683c77d4ff0",
        "is_bot": false
    },
    "updated_by_user_id": "97abf1ee-6308-11e7-9d09-00163ec04961"
}

Create a new experiment

POST /api/v5/orgs/<organization_id>/experiments

Attribute Required
is_draft required

If is_draft is true, all of the following attributes are optional. Otherwise, if is_draft is false:

Attribute Required
goal_id required
groups see groups below
name required
room_ids required
starts_at required
ends_at optional
group_reset_interval optional
Attribute Required
name required
percentage required
group_type required
treatment optional

This endpoint returns:

  • 201 if a new experiment was created successfully
  • 400 if required attributes are missing
  • 400 if payload attributes are in wrong format
  • 401 if you are not authenticated
  • 403 if you do not have access to the organization
  • 403 if you do not have active subscription
  • 403 if you do not have settings permission

Example payload:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
{
    "goal_id": "0a534844-869a-11e9-8dcc-27f1dd2d9764",
    "groups": [
        {
            "name": "A",
            "percentage": 90,
            "group_type": "control",
            "treatment": {
                "name": "Rule-A",
                "actions": [],
                "conditions": []
            }
        },
        {
            "name": "B",
            "percentage": 10,
            "group_type": "user_defined",
            "treatment": {
                "name": "Rule-B",
                "actions": [],
                "conditions": []
            }
        },
    ],
    "group_reset_interval": "full",
    "name": "Hello experiment",
    "starts_at": "2019-05-08T13:02:35.123Z",
    "ends_at": "2019-05-22T13:02:35.123Z"
}

Update experiment

PUT /api/v5/orgs/<organization_id>/experiments/<experiment_id>

Attribute Required
is_draft required

If is_draft is true, all of the following attributes are optional. Otherwise, if is_draft is false:

goal_id | read-only if experiment has already started (starts_at has passed), else required groups | see groups below name | required room_ids | read-only if experiment has already started (starts_at has passed), else required starts_at | read-only if experiment has already started (starts_at has passed), else required ends_at | optional group_reset_interval | read-only if experiment has already started (starts_at has passed), else optional

Attribute Required
name required
percentage required
group_type read-only if experiment has already started (starts_at has passed), else optional
treatment read-only if experiment has already started (starts_at has passed), else optional

This endpoint returns:

  • 200 if the experiment was updated successfully
  • 400 if payload attributes are in wrong format
  • 400 if experiment has ended
  • 400 if ends_at is before starts_at
  • 400 if trying to update is_draft from false to true
  • 401 if you are not authenticated
  • 403 if you do not have access to the organization
  • 403 if you do not have active subscription
  • 403 if you do not have settings permission
  • 404 if the experiment was not found

Delete experiment

DELETE /api/v5/orgs/<organization_id>/experiments/<experiment_id>

This endpoint returns:

  • 204 if the experiment was deleted successfully
  • 401 if you are not authenticated
  • 403 if you do not have access to the organization
  • 403 if you do not have active subscription
  • 403 if you do not have settings permission
  • 404 if the experiment was not found