Experiment types
Block, page, flag and code tests, what each one changes, and how each one counts exposures.
Every test has a kind. The kind decides what changes between versions, where the versions live, and how a visit is counted. Editors pick the kind when they create a test in Tests → New test.
| Kind | Admin card | What changes | Versions live in | Counted when |
|---|---|---|---|---|
block | A block on a page | One section of a page | Version slots inside an Experiment block | The block renders (render) or scrolls into view (viewport) |
page | A whole page | The whole page | Separate published pages, each at its own address | The page shows (pageview) |
flag | A setting | Settings at a flag point your code reads | The test's own settings | Your code calls expose() (manual) |
code | — | Anything a developer codes | Your code | Your code calls expose() (manual) |
Each test has an Original (the control) and up to three variants, Variant A, Variant B and Variant C, plus an optional holdout. Group percentages are whole numbers that total exactly 100.
Block tests
A block test wraps one part of a page in an Experiment block. The block holds one slot per version; the Original slot keeps the current content. Editors build versions in place, in Puck or in a Payload blocks field, and can copy the Original into a version to start from it.
On the site, resolveExperimentData (Puck) or resolveExperimentBlocks (blocks field) replaces the Experiment block with the visitor's version. Visitors outside the test and visitors when no test is running see the Original exactly as it renders without the plugin.
A block test runs wherever its block is placed, so it cannot have a page-path targeting rule. It can count visitors or signed-in users.
Set up: Puck or Payload blocks.
Page tests
A page test compares whole pages. The Original is an existing published page; each variant is another published page with its own address, usually made with Payload's Duplicate action. Requests to any page in the test are rewritten by the proxy to the visitor's assigned page, so every visitor sees one consistent version at the Original's address.
Page tests always count visitors, never signed-in users. While a test runs, variant pages reached at their own address are marked noindex with a canonical link to the Original.
Set up: Whole pages.
Flag tests
A flag point is a stable place in your code that reads settings, registered once in the plugin options. Editors can then run any number of tests on it without code changes: each version is a set of values for the point's settings, such as a heading or a button label.
When no test runs, the point returns its saved winner defaults, or its defaultParams. Reading a flag point never counts a visit; your code calls expose() when it delivers that version.
Set up: Flag points.
Code tests
A code test is read by its short name with useExperiment(slug) in a client component, or from assignments.bySlug on the server. A developer implements each version in code and calls expose() when it shows. Creating code tests and changing the analysis method need the access.admin permission.
There is no Make it the content for code tests. Instead, after deciding, a developer hard-codes the winner, and you confirm that with typed confirmation to archive the test.
Statuses
A test moves through these statuses:
| Status | Meaning |
|---|---|
draft | Being set up. Nobody sees it. |
running | Assigning and counting people. |
stopped | Everyone sees the Original. Starting again begins a new round with fresh results. |
decided | A winner was chosen and is shown to everyone (except an active holdout). |
archived | Closed. A page test's archived routing can keep sending losing addresses to the winner. |
Each start begins a new round. A round freezes its settings (groups, goals, analysis method, sequential tuning value, revenue caps) when it starts, so results never change because a setting changed later.