Experiments and attribution

Experiments provide one shared way to compare popup variants, campaign split branches, and other compatible content. A popup or campaign split keeps the same deterministic assignment behaviour during migration; its shared experiment adds durable reporting and a permanent report link.

Configure a version

An experiment has two to five variants, exactly 10,000 allocation basis points in total, and at most one control variant. A control can represent no message or no content. Choose whether assignment is by subscriber or by journey:

  • Subscriber assignment keeps the same person in the same variant.
  • Journey assignment keeps one campaign journey in the same branch while allowing a later journey to be independently allocated.

Assignment is deterministic and durable. It is based on the immutable experiment version and assignment ID, so a callback retry does not move a customer to another variant.

Choose the audience, primary goal, optional secondary goals, value source, and attribution window before starting. A version can reference compatible content and goals, including value from a constant, event property, payment, or CRM deal where configured.

Versions and lifecycle

An experiment moves through draft, running, paused, completed, cancelled, and archived states. Starting runs the current immutable version. Pausing stops the active lifecycle without changing its assignment definition; completed or cancelled experiments can then be archived.

Once a version has exposure, changing its content or allocation creates a new version instead of editing the exposed one. Keep the version picker on the report in mind when comparing historic results. Winner selection is a manual, one-time action and remains visible after archival; it is not an automatic significance decision.

Facts and attribution

The report ladder contains assignments, exposure, delivery, open, click, reply, conversion, unsubscribe, and complaint. These facts use a durable source event ID, so repeated callbacks are idempotent. Revenue attribution is recorded separately from facts and uses signed 64-bit minor units with an ISO 4217 currency, for example 1250 + EUR for €12.50.

Revenue is grouped by currency. The product never silently adds EUR, RSD, and USD together. Provide a separate, timestamped conversion process before making any cross-currency comparison.

Choose one attribution model for a report:

  • First touch attributes an eligible conversion to the first touch.
  • Last touch attributes it to the latest eligible touch.
  • Assignment attributes it to the experiment assignment.

The selected model and the immutable version's attribution window are shown in the response and UI. The report does not combine models in one total.

Warnings and cohorts

Warnings are deterministic data-quality checks, not statistical significance claims. They cover an incomplete experiment, a variant with fewer than 30 exposures, assignments without exposures, materially imbalanced allocation after sufficient assignment volume, and clicks or conversions exceeding exposures. Investigate instrumentation or delivery before choosing a winner.

Use the cohort drawer to page through assignments by variant. The list uses a cursor and canonical customer identity. Segment filtering in a report uses the currently published membership for the selected segment.

Compatibility with popups and campaigns

Multi-variant popups get a shared experiment only after their legacy facts are backfilled. A previously shown popup variant wins for that returning visitor, so the migration does not change content mid-journey. Popup statistics retain their existing response shape and fall back to legacy counters when parity is not proven.

Campaign split links record a cutover time. Journeys that began before that time stay on the original deterministic split. A shared report becomes ready only after those legacy journeys finish, so the system does not invent historical branch assignments.

API and MCP examples

The REST report needs a PAT with experiments:read and the Analytics bot right. The range is bounded RFC3339 UTC and the JSON does not contain a real token or customer payload.

curl -X POST "$BASE_URL/api/bots/$BOT_ID/experiments/$EXPERIMENT_ID/report" \
  -H "Authorization: Bearer $PAT" \
  -H "Content-Type: application/json" \
  -d '{
    "version": 2,
    "attribution_model": "assignment",
    "from": "2026-07-01T00:00:00Z",
    "to": "2026-08-01T00:00:00Z",
    "segment_id": "00000000-0000-4000-8000-000000000000"
  }'

400 means an invalid report request. 404 means the experiment or explicitly requested version is unavailable in this bot. 409 applies to an optimistic update conflict or a winner-selection conflict, not to a missing report version. Read an assignment cohort with version, optional variant, limit, and the returned cursor as query parameters.

MCP exposes read-only shared experiment tools with the same experiments:read scope:

{"name":"get_experiment_report","arguments":{"bot_id":"<bot-id>","experiment_id":"<experiment-id>","version":2}}

list_experiments and get_experiment are also available. MCP does not start, pause, edit, choose a winner, create assignments, or write facts.

Related pages