Skip to content

Building a custom pipeline

This guide walks through creating a pipeline definition from scratch: choosing event filters, counting distinct visitors per step, referencing value sets, and adding metrics and dimensions.

Prerequisites: a valid API token and your organization_id.
Base URL: https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline

Step 1 — Discover your event data

Before writing filters, find out what vendor, category, and action combinations actually exist in your data:

1
2
3
4
5
6
7
8
POST https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline/events
Content-Type: application/json
Authorization: Bearer <token>

{
  "start": "2024-01-01T00:00:00Z",
  "end": "2024-01-31T23:59:59Z"
}

The response lists every distinct (vendor, category, action) triple recorded in that window. Use these values in your pipeline steps. Add "include_blacklisted_categories": true if you also want internal/system event types.

Step 2 — Create a pipeline definition

A definition needs a name and at least one step. Each step needs a name and at least one event filter.

Minimal two-step example — unique visitors who started vs clicked the widget:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
POST https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline/definitions/
Content-Type: application/json
Authorization: Bearer <token>

{
  "name": "Widget engagement",
  "description": "Visitors who started vs clicked the widget",
  "steps": [
    {
      "name": "Widget started",
      "count_field": "visitor_id",
      "events": [
        { "category": "widget", "action": "start" }
      ]
    },
    {
      "name": "Widget clicked",
      "count_field": "visitor_id",
      "events": [
        { "category": "widget", "action": "click" }
      ]
    }
  ]
}

The response includes the created definition. Copy the id — you need it to run counts.

1
2
3
4
5
6
{
  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "name": "Widget engagement",
  "is_predefined": false,
  "steps": [ ... ]
}

count_field defaults to visitor_id (distinct visitors). You can also use session_id (distinct visits) or event_id (total event occurrences).

Naming and tagging pipelines

Two optional fields help you organize pipelines as you create more of them:

  • slug — a stable, unique-per-organization identifier (lowercase letters, digits, hyphens, underscores) you can use to fetch the pipeline later without keeping track of its UUID. Cannot collide with the predefined slugs impressions or shopping-cart.
  • tags — a list of free-form labels you can filter on when listing pipelines.
1
2
3
4
5
6
{
  "name": "Widget engagement",
  "slug": "widget-engagement",
  "tags": ["marketing", "widget"],
  "steps": [ ... ]
}

Look the pipeline up later by slug instead of UUID:

1
GET https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline/definitions/by-slug/widget-engagement/

Or filter the list endpoint by tag (repeat tag for multiple tags; matching is AND):

1
GET https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline/definitions/?tag=marketing

Step 3 — Get counts

1
2
3
4
5
6
7
8
9
POST https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline/counts
Content-Type: application/json
Authorization: Bearer <token>

{
  "pipeline_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "start": "2024-01-01T00:00:00Z",
  "end": "2024-01-31T23:59:59Z"
}

Response:

1
2
3
4
5
6
7
8
{
  "pipeline_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "pipeline_name": "Widget engagement",
  "steps": [
    { "step_name": "Widget started", "count": 1250 },
    { "step_name": "Widget clicked", "count": 480  }
  ]
}

Each count is the number of distinct values of the step's count_field that matched that step's filters in the period. Steps are independent — there is no funnel requirement.

Value sets

Value sets let you store named lists of IDs (e.g. interaction IDs, goal IDs) and reference them in event filters as $name. When the pipeline runs, the reference is replaced by the list for your organization. This means you can update which IDs are tracked without modifying the pipeline definition.

Create a value set

1
2
3
4
5
6
7
8
POST https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline/value-sets/
Content-Type: application/json
Authorization: Bearer <token>

{
  "name": "interaction-ids",
  "values": ["int-abc-123", "int-def-456"]
}

Repeat for any other lists you need (e.g. goal-ids). To update values later, use PATCH on the value set's URL — the pipeline definition stays unchanged.

Use a value set in a filter

Reference the value set by prefixing its name with $:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
{
  "name": "Impressions",
  "count_field": "visitor_id",
  "events": [
    {
      "category": "widget",
      "action": "start",
      "label": "$interaction-ids"
    }
  ]
}

At query time, $interaction-ids is resolved to ["int-abc-123", "int-def-456"] for your organization, and the step only counts events whose label matches one of those IDs.

Full example — impressions → engagement → goals, filtered by value sets:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
{
  "name": "Impressions to goals (filtered)",
  "steps": [
    {
      "name": "Impressions",
      "count_field": "visitor_id",
      "events": [{ "category": "widget", "action": "start", "label": "$interaction-ids" }]
    },
    {
      "name": "Engaged",
      "count_field": "visitor_id",
      "events": [{ "category": "widget", "action": "click", "label": "$interaction-ids" }]
    },
    {
      "name": "Goals reached",
      "count_field": "visitor_id",
      "events": [{ "category": "goal", "action": "trigger", "label": "$goal-ids" }]
    }
  ]
}

Filtering by room

Add a room_ids field to any event filter to restrict that step to specific rooms. A row matches if any of the event's room IDs appear in the filter list.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
{
  "name": "Chat started in selected rooms",
  "count_field": "visitor_id",
  "events": [
    {
      "category": "chat",
      "action": "start",
      "room_ids": ["room-uuid-1", "room-uuid-2"]
    }
  ]
}

You can also use a value set reference ("room_ids": "$my-room-ids") to manage the list externally.

Filtering by event properties

The properties field matches entries in an event's properties array. All entries you specify must be present (AND logic). Unlike other fields, properties does not support value set references.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
{
  "name": "Cart Purchased with Real Chat",
  "count_field": "visitor_id",
  "events": [
    {
      "vendor": "com.giosg.chat",
      "category": "shopping-cart",
      "action": "purchased",
      "properties": ["with_real_conversation=true"]
    }
  ]
}

Adding metrics to a step

A metric is an optional sub-query on a step that returns a grouped breakdown. Metrics are only evaluated by the step-detail endpoint — they are ignored by the counts endpoint.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
{
  "name": "Goals reached",
  "count_field": "visitor_id",
  "events": [{ "category": "goal", "action": "trigger", "label": "$goal-ids" }],
  "metrics": [
    {
      "name": "Users by goal",
      "measure": {
        "field": "visitor_id",
        "aggregations": ["count_distinct"]
      },
      "group_by": ["label"]
    }
  ]
}

Translating IDs to names with mapping_functions

If the group_by field contains opaque IDs (goal IDs, interaction IDs), add mapping_functions to have the step-detail endpoint translate them to human-readable names:

1
2
3
4
5
6
{
  "name": "Users by goal",
  "measure": { "field": "visitor_id", "aggregations": ["count_distinct"] },
  "group_by": ["label"],
  "mapping_functions": { "label": "f$goals" }
}

"f$goals" translates goal IDs to goal names; "f$interactions" translates interaction IDs to interaction names. Unknown IDs are kept as-is.

Hinting at single-period display with single_period

Some metrics (e.g. a duration or count aggregated over a whole case) don't make sense as a multi-period comparison in a UI — the events involved usually fall within one short timespan. Set "single_period": true on such a metric so consumers of the API know to render it as a single value instead of a period-over-period table:

1
2
3
4
5
6
{
  "name": "Duration by case",
  "measure": { "field": "value", "aggregations": ["sum", "avg", "min", "max"] },
  "group_by": ["dim3"],
  "single_period": true
}

It defaults to false and is returned as-is wherever the metric appears — on the pipeline definition and on the step object in step-detail responses.

Empty results

If a step has no matching events in the requested range, every metric in the step-detail response is [] — including ungrouped metrics like a plain sum/avg with no group_by, which previously could return a single all-null row.

Defining dimensions

Dimensions let you slice all step counts at query time by a shared property (e.g. country, city) without modifying the definition. You declare how each step maps to the filterable property, and then pass filter values at request time.

Direct field mapping

Each step filters directly on the named event field. Use this when the field carries the values you want on every step. For a two-step widget pipeline where label holds the interaction ID on both steps:

1
2
3
4
5
6
7
8
9
{
  "name": "interaction",
  "label": "Interaction",
  "step_mappings": [
    { "step_index": 0, "field": "label" },
    { "step_index": 1, "field": "label" }
  ],
  "pageview_cohort_from_step": 0
}

pageview_cohort_from_step is optional — it tells the API which step to use when filtering pageview counts by this dimension.

Cohort mapping

When a step's events don't carry the field you want to filter by, use cohort_from_step to inherit sessions from a step that does. In a three-step pipeline — Impressions (widget), Engaged (widget), Goals reached (goal) — widget events use label for interaction IDs, but goal events use label for goal IDs. The goal step can't be filtered directly by interaction ID, so it inherits sessions from step 0:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
{
  "name": "interaction",
  "label": "Interaction",
  "step_mappings": [
    { "step_index": 0, "field": "label" },
    { "step_index": 1, "field": "label" },
    { "step_index": 2, "cohort_from_step": 0 }
  ],
  "pageview_cohort_from_step": 0
}

Step 2 doesn't filter directly on label; instead, it only counts sessions whose session_id matched the interaction filter on step 0.

Full create request with dimensions

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
{
  "name": "Widget engagement",
  "pageview_source_field": "referrer_medium",
  "steps": [
    {
      "name": "Impressions",
      "count_field": "visitor_id",
      "events": [{ "category": "widget", "action": "start" }]
    },
    {
      "name": "Engaged",
      "count_field": "visitor_id",
      "events": [{ "category": "widget", "action": "click" }]
    },
    {
      "name": "Goals reached",
      "count_field": "visitor_id",
      "events": [{ "category": "goal" }]
    }
  ],
  "dimensions": [
    {
      "name": "interaction",
      "label": "Interaction",
      "step_mappings": [
        { "step_index": 0, "field": "label" },
        { "step_index": 1, "field": "label" },
        { "step_index": 2, "cohort_from_step": 0 }
      ],
      "pageview_cohort_from_step": 0
    },
    {
      "name": "goal",
      "label": "Goal",
      "step_mappings": [
        { "step_index": 0, "cohort_from_step": 2 },
        { "step_index": 1, "cohort_from_step": 2 },
        { "step_index": 2, "field": "label" }
      ]
    }
  ]
}

Full dimensions must have exactly one step_mapping per pipeline step (indexes 0, 1, ..., N-1). For dimensions that only apply to some steps, see partial dimensions in the reference.

Syncing a pipeline from a case funnel

If you already have a giosg case funnel, you can create (or refresh) a matching pipeline definition without building steps by hand:

1
2
3
4
5
6
7
POST https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline/funnels/<funnel_id>/sync
Content-Type: application/json
Authorization: Bearer <token>

{
  "mode": "update"
}
  • mode: "update" (default) — updates the newest pipeline tagged funnel:<funnel_id>, or creates one if none exists. Returns 200 on update, 201 on create.
  • mode: "create" — always creates a new pipeline definition. Returns 201.

An empty body {} is the same as {"mode": "update"}.

What you get back — a normal pipeline definition (same shape as create/retrieve):

  • tags includes funnel:<funnel_id>, which is how later syncs find the pipeline again.
  • One step per funnel stage, ordered by stage order, using category=case_management and the stage id as label.
  • Each step gets Duration by case, Count by case, and Duration total metrics (terminal/closed stages get Count by case only). The by-case metrics are marked single_period: true.
  • case and chat dimensions attached automatically, mapped on every step.

Note

Funnel-generated metric names no longer include the stage name. If you synced a pipeline before this change, running the sync again (mode: "update" or "create") picks up the new names automatically.

Find synced pipelines later with:

1
GET https://api.giosg.com/api/objectives/v1/orgs/<organization_id>/pipeline/definitions/?tag=funnel:<funnel_id>

Then use the returned id with the counts or step-detail endpoints as usual. See Sync from a case funnel in the reference for the full mapping rules and error responses.

Next steps