Experiments API

Experiments¶

An experiment represents an A/B test. It has a goal and multiple groups with different treatments. Experiments can be limited duration or indefinite.

An experiment resource has following attributes:

Attribute	Type	Description
`id`	UUID	An unique ID for the experiment
`is_draft`	boolean	Whether the experiment is a draft or published. Once published, the experiment cannot be changed back to draft
`created_at`	datetime	Create time of the experiment
`created_by_user`	object	The user who created this experiment
`created_by_user_id`	string	ID of the user who created this experiment
`updated_at`	datetime	Update time of the experiment
`updated_by_user`	object	The user who last updated this experiment
`updated_by_user_id`	string	ID of the user who last updated this experiment
`organization_id`	UUID	ID of the organization that owns this experiment
`organization`	object	The organization that owns this experiment
`goal_id`	UUID	ID of the goal of this experiment
`goal`	object	The goal of this experiment
`groups`	array of objects	List of the experiment's groups
`name`	string	The name of the experiment
`starts_at`	datetime	The start time of the experiment
`ends_at`	datetime	The end time of the experiment
`group_reset_interval`	string	When to reset group's visitors, one of: `full` (default), `1w` or `1d`
`rooms`	array of objects	List of rooms where the experiment treatments can be triggered. If the list is empty, treatment can be triggered in any of the organization's rooms
`room_ids`	array of UUIDs	List of room IDs where the experiment treatments can be triggered. If the list is empty, treatment can be triggered in any of the organization's rooms

Experiment groups¶

Attribute	Type	Description
`id`	UUID	The unique identifier of this experiment group
`created_at`	datetime	Create time of the group
`updated_at`	datetime	Update time of the group
`order`	integer	The index of this group in the order of groups within the experiment
`name`	string	Name of this group
`percentage`	integer	The percentage this group has within the experiments group distribution. Percentages of groups within a experiment must add up to `100`
`group_type`	string	Group type. One of `control`, `user_defined`, `target_algorithm`, `target_random` or `null`
`treatment`	object	Rule that is ran for visitors in this group

List experiments¶

GET /api/v5/orgs/<organization_id>/experiments

Retrieve details of an experiment¶

GET /api/v5/orgs/<organization_id>/experiments/<experiment_id>

Example of experiment details:

{
    "is_draft": false,
    "created_at": "2019-02-13T11:31:36.042",
    "created_by_user": {
        "id": "97abf1ee-6308-11e7-9d09-00163ec04961",
        "first_name": "John",
        "last_name": "Doe",
        "full_name": "John Doe",
        "organization_id": "aabefa3e-1c8e-4690-99e4-2683c77d4ff0",
        "is_bot": false
    },
    "created_by_user_id": "97abf1ee-6308-11e7-9d09-00163ec04961",
    "goal_id": "0a534844-869a-11e9-8dcc-27f1dd2d9764",
    "goal": {
        "id": "0a534844-869a-11e9-8dcc-27f1dd2d9764",
        "name": "Reached checkout page",
        "organization": {
            "id": "aabefa3e-1c8e-4690-99e4-2683c77d4ff0",
            "name": "Company Inc."
        },
        "organization_id": "aabefa3e-1c8e-4690-99e4-2683c77d4ff0"
    },
    "groups": [
        {
            "name": "A",
            "percentage": 90,
            "group_type": "control",
            "treatment": {}
        },
        {
            "name": "B",
            "percentage": 10,
            "group_type": "user_defined",
            "treatment": {}
        }
    ],
    "group_reset_interval": "full",
    "name": "Hello experiment",
    "starts_at": "2019-05-08T13:02:35.123Z",
    "ends_at": "2019-05-22T13:02:35.123Z",
    "updated_at": "2019-05-07T11:00:00.000Z",
    "updated_by_user": {
        "id": "97abf1ee-6308-11e7-9d09-00163ec04961",
        "first_name": "John",
        "last_name": "Doe",
        "full_name": "John Doe",
        "organization_id": "aabefa3e-1c8e-4690-99e4-2683c77d4ff0",
        "is_bot": false
    },
    "updated_by_user_id": "97abf1ee-6308-11e7-9d09-00163ec04961"
}

Create a new experiment¶

POST /api/v5/orgs/<organization_id>/experiments

Attribute	Required
`is_draft`	required

If is_draft is true, all of the following attributes are optional. Otherwise, if is_draft is false:

Attribute	Required
`goal_id`	required
`groups`	see `groups` below
`name`	required
`room_ids`	required
`starts_at`	required
`ends_at`	optional
`group_reset_interval`	optional

Attribute	Required
`name`	required
`percentage`	required
`group_type`	required
`treatment`	optional

This endpoint returns:

201 if a new experiment was created successfully
400 if required attributes are missing
400 if payload attributes are in wrong format
401 if you are not authenticated
403 if you do not have access to the organization
403 if you do not have active subscription
403 if you do not have settings permission

Example payload:

{
    "goal_id": "0a534844-869a-11e9-8dcc-27f1dd2d9764",
    "groups": [
        {
            "name": "A",
            "percentage": 90,
            "group_type": "control",
            "treatment": {
                "name": "Rule-A",
                "actions": [],
                "conditions": []
            }
        },
        {
            "name": "B",
            "percentage": 10,
            "group_type": "user_defined",
            "treatment": {
                "name": "Rule-B",
                "actions": [],
                "conditions": []
            }
        },
    ],
    "group_reset_interval": "full",
    "name": "Hello experiment",
    "starts_at": "2019-05-08T13:02:35.123Z",
    "ends_at": "2019-05-22T13:02:35.123Z"
}

Update experiment¶

PUT /api/v5/orgs/<organization_id>/experiments/<experiment_id>

Attribute	Required
`is_draft`	required

If is_draft is true, all of the following attributes are optional. Otherwise, if is_draft is false:

goal_id | read-only if experiment has already started (starts_at has passed), else required groups | see groups below name | required room_ids | read-only if experiment has already started (starts_at has passed), else required starts_at | read-only if experiment has already started (starts_at has passed), else required ends_at | optional group_reset_interval | read-only if experiment has already started (starts_at has passed), else optional

Attribute	Required
`name`	required
`percentage`	required
`group_type`	read-only if experiment has already started (`starts_at` has passed), else optional
`treatment`	read-only if experiment has already started (`starts_at` has passed), else optional

This endpoint returns:

200 if the experiment was updated successfully
400 if payload attributes are in wrong format
400 if experiment has ended
400 if ends_at is before starts_at
400 if trying to update is_draft from false to true
401 if you are not authenticated
403 if you do not have access to the organization
403 if you do not have active subscription
403 if you do not have settings permission
404 if the experiment was not found

Delete experiment¶

DELETE /api/v5/orgs/<organization_id>/experiments/<experiment_id>

This endpoint returns:

204 if the experiment was deleted successfully
401 if you are not authenticated
403 if you do not have access to the organization
403 if you do not have active subscription
403 if you do not have settings permission
404 if the experiment was not found