Skip to content

Building a custom pipeline

This guide walks through creating a pipeline definition from scratch: choosing event filters, counting distinct visitors per step, referencing value sets, and adding metrics and dimensions.

Prerequisites: a valid API token and your organization_id.
Base URL: https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline

Step 1 — Discover your event data

Before writing filters, find out what vendor, category, and action combinations actually exist in your data:

1
2
3
4
5
6
7
8
POST https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline/events
Content-Type: application/json
Authorization: Bearer <token>

{
  "start": "2024-01-01T00:00:00Z",
  "end": "2024-01-31T23:59:59Z"
}

The response lists every distinct (vendor, category, action) triple recorded in that window. Use these values in your pipeline steps. Add "include_blacklisted_categories": true if you also want internal/system event types.

Step 2 — Create a pipeline definition

A definition needs a name and at least one step. Each step needs a name and at least one event filter.

Minimal two-step example — unique visitors who started vs clicked the widget:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
POST https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline/definitions/
Content-Type: application/json
Authorization: Bearer <token>

{
  "name": "Widget engagement",
  "description": "Visitors who started vs clicked the widget",
  "steps": [
    {
      "name": "Widget started",
      "count_field": "visitor_id",
      "events": [
        { "category": "widget", "action": "start" }
      ]
    },
    {
      "name": "Widget clicked",
      "count_field": "visitor_id",
      "events": [
        { "category": "widget", "action": "click" }
      ]
    }
  ]
}

The response includes the created definition. Copy the id — you need it to run counts.

1
2
3
4
5
6
{
  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "name": "Widget engagement",
  "is_predefined": false,
  "steps": [ ... ]
}

count_field defaults to visitor_id (distinct visitors). You can also use session_id (distinct visits) or event_id (total event occurrences).

Step 3 — Get counts

1
2
3
4
5
6
7
8
9
POST https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline/counts
Content-Type: application/json
Authorization: Bearer <token>

{
  "pipeline_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "start": "2024-01-01T00:00:00Z",
  "end": "2024-01-31T23:59:59Z"
}

Response:

1
2
3
4
5
6
7
8
{
  "pipeline_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "pipeline_name": "Widget engagement",
  "steps": [
    { "step_name": "Widget started", "count": 1250 },
    { "step_name": "Widget clicked", "count": 480  }
  ]
}

Each count is the number of distinct values of the step's count_field that matched that step's filters in the period. Steps are independent — there is no funnel requirement.

Value sets

Value sets let you store named lists of IDs (e.g. interaction IDs, goal IDs) and reference them in event filters as $name. When the pipeline runs, the reference is replaced by the list for your organization. This means you can update which IDs are tracked without modifying the pipeline definition.

Create a value set

1
2
3
4
5
6
7
8
POST https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline/value-sets/
Content-Type: application/json
Authorization: Bearer <token>

{
  "name": "interaction-ids",
  "values": ["int-abc-123", "int-def-456"]
}

Repeat for any other lists you need (e.g. goal-ids). To update values later, use PATCH on the value set's URL — the pipeline definition stays unchanged.

Use a value set in a filter

Reference the value set by prefixing its name with $:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
{
  "name": "Impressions",
  "count_field": "visitor_id",
  "events": [
    {
      "category": "widget",
      "action": "start",
      "label": "$interaction-ids"
    }
  ]
}

At query time, $interaction-ids is resolved to ["int-abc-123", "int-def-456"] for your organization, and the step only counts events whose label matches one of those IDs.

Full example — impressions → engagement → goals, filtered by value sets:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
{
  "name": "Impressions to goals (filtered)",
  "steps": [
    {
      "name": "Impressions",
      "count_field": "visitor_id",
      "events": [{ "category": "widget", "action": "start", "label": "$interaction-ids" }]
    },
    {
      "name": "Engaged",
      "count_field": "visitor_id",
      "events": [{ "category": "widget", "action": "click", "label": "$interaction-ids" }]
    },
    {
      "name": "Goals reached",
      "count_field": "visitor_id",
      "events": [{ "category": "goal", "action": "trigger", "label": "$goal-ids" }]
    }
  ]
}

Filtering by room

Add a room_ids field to any event filter to restrict that step to specific rooms. A row matches if any of the event's room IDs appear in the filter list.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
{
  "name": "Chat started in selected rooms",
  "count_field": "visitor_id",
  "events": [
    {
      "category": "chat",
      "action": "start",
      "room_ids": ["room-uuid-1", "room-uuid-2"]
    }
  ]
}

You can also use a value set reference ("room_ids": "$my-room-ids") to manage the list externally.

Filtering by event properties

The properties field matches entries in an event's properties array. All entries you specify must be present (AND logic). Unlike other fields, properties does not support value set references.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
{
  "name": "Cart Purchased with Real Chat",
  "count_field": "visitor_id",
  "events": [
    {
      "vendor": "com.giosg.chat",
      "category": "shopping-cart",
      "action": "purchased",
      "properties": ["with_real_conversation=true"]
    }
  ]
}

Adding metrics to a step

A metric is an optional sub-query on a step that returns a grouped breakdown. Metrics are only evaluated by the step-detail endpoint — they are ignored by the counts endpoint.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
{
  "name": "Goals reached",
  "count_field": "visitor_id",
  "events": [{ "category": "goal", "action": "trigger", "label": "$goal-ids" }],
  "metrics": [
    {
      "name": "Users by goal",
      "measure": {
        "field": "visitor_id",
        "aggregations": ["count_distinct"]
      },
      "group_by": ["label"]
    }
  ]
}

Translating IDs to names with mapping_functions

If the group_by field contains opaque IDs (goal IDs, interaction IDs), add mapping_functions to have the step-detail endpoint translate them to human-readable names:

1
2
3
4
5
6
{
  "name": "Users by goal",
  "measure": { "field": "visitor_id", "aggregations": ["count_distinct"] },
  "group_by": ["label"],
  "mapping_functions": { "label": "f$goals" }
}

"f$goals" translates goal IDs to goal names; "f$interactions" translates interaction IDs to interaction names. Unknown IDs are kept as-is.

Defining dimensions

Dimensions let you slice all step counts at query time by a shared property (e.g. country, city) without modifying the definition. You declare how each step maps to the filterable property, and then pass filter values at request time.

Direct field mapping

Each step filters directly on the named event field. Use this when the field carries the values you want on every step. For a two-step widget pipeline where label holds the interaction ID on both steps:

1
2
3
4
5
6
7
8
9
{
  "name": "interaction",
  "label": "Interaction",
  "step_mappings": [
    { "step_index": 0, "field": "label" },
    { "step_index": 1, "field": "label" }
  ],
  "pageview_cohort_from_step": 0
}

pageview_cohort_from_step is optional — it tells the API which step to use when filtering pageview counts by this dimension.

Cohort mapping

When a step's events don't carry the field you want to filter by, use cohort_from_step to inherit sessions from a step that does. In a three-step pipeline — Impressions (widget), Engaged (widget), Goals reached (goal) — widget events use label for interaction IDs, but goal events use label for goal IDs. The goal step can't be filtered directly by interaction ID, so it inherits sessions from step 0:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
{
  "name": "interaction",
  "label": "Interaction",
  "step_mappings": [
    { "step_index": 0, "field": "label" },
    { "step_index": 1, "field": "label" },
    { "step_index": 2, "cohort_from_step": 0 }
  ],
  "pageview_cohort_from_step": 0
}

Step 2 doesn't filter directly on label; instead, it only counts sessions whose session_id matched the interaction filter on step 0.

Full create request with dimensions

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
{
  "name": "Widget engagement",
  "pageview_source_field": "referrer_medium",
  "steps": [
    {
      "name": "Impressions",
      "count_field": "visitor_id",
      "events": [{ "category": "widget", "action": "start" }]
    },
    {
      "name": "Engaged",
      "count_field": "visitor_id",
      "events": [{ "category": "widget", "action": "click" }]
    },
    {
      "name": "Goals reached",
      "count_field": "visitor_id",
      "events": [{ "category": "goal" }]
    }
  ],
  "dimensions": [
    {
      "name": "interaction",
      "label": "Interaction",
      "step_mappings": [
        { "step_index": 0, "field": "label" },
        { "step_index": 1, "field": "label" },
        { "step_index": 2, "cohort_from_step": 0 }
      ],
      "pageview_cohort_from_step": 0
    },
    {
      "name": "goal",
      "label": "Goal",
      "step_mappings": [
        { "step_index": 0, "cohort_from_step": 2 },
        { "step_index": 1, "cohort_from_step": 2 },
        { "step_index": 2, "field": "label" }
      ]
    }
  ]
}

Full dimensions must have exactly one step_mapping per pipeline step (indexes 0, 1, ..., N-1). For dimensions that only apply to some steps, see partial dimensions in the reference.

Next steps