payload-experiments
A/B testing for Payload and Next.js. Your data stays in your database.
A/B testing inside Payload. Your data stays yours.
Editors test a block, a whole page or a setting, pick a goal, and read a plain-English verdict in the Payload admin. Exposures, conversions and results live in your own Postgres, analysed by a TypeScript port of GrowthBook's gbstats.
Prerelease. Expect API changes and database migrations. Read the known limitations before adopting it.
pnpm add @delmaredigital/payload-experiments@rcAlso available: The statistics engine on its own: a TypeScript port of GrowthBook’s gbstats.
pnpm add @delmaredigital/experiments-statsStart with your existing Payload app.
These files show the integration points. Follow the installation guide for peers and the full documentation for the remaining application setup.
import { Pool } from 'pg'
import {
defineExperimentsProxyRuntime, createPostgresManifestLoader,
} from '@delmaredigital/payload-experiments/runtime/proxy'
const pool = new Pool({
connectionString: process.env.DATABASE_URI,
connectionTimeoutMillis: 5_000,
})
export const proxyRuntime = defineExperimentsProxyRuntime({
secret: process.env.XP_SECRET!,
apiRoute: '/api',
loadManifest: createPostgresManifestLoader({ pool }),
})import { defineExperimentsServerRuntime } from
'@delmaredigital/payload-experiments/runtime/server'
import { proxyRuntime } from './proxyRuntime'
export const serverRuntime = defineExperimentsServerRuntime(proxyRuntime, {
getPayload: async () => {
const { getPayload } = await import('payload')
const { default: config } = await import('@payload-config')
return getPayload({ config })
},
})import { createExperimentsPlugin } from '@delmaredigital/payload-experiments'
import { serverRuntime } from '@/experiments/serverRuntime'
import { canManageTests } from '@/access/tests'
import { sharedRateLimit } from '@/experiments/rateLimit'
// Add to your existing plugins array. Supply both host callbacks.
createExperimentsPlugin({
runtime: serverRuntime,
access: { manage: canManageTests },
events: { rateLimit: sharedRateLimit },
})import { createExperimentProxy } from '@delmaredigital/payload-experiments/proxy'
import { proxyRuntime } from '@/experiments/proxyRuntime'
export const proxy = createExperimentProxy(proxyRuntime)What it does
- Test what editors already edit
- Blocks, whole pages and registered settings, plus code tests for developers. Set up once; no code per test.
- Only real exposures count
- Results include only people who actually saw a version, with a traffic-split (SRM) check on every result.
- Goals with guardrails
- A main goal, guardrail metrics that can veto a win, and click goals editors attach to buttons.
- Statistics you can check
- A function-by-function port of gbstats, matched against the Python original on 11,774 cases. The default sequential method stays valid however often you look.
- Holdouts and layers
- Keep a group out of every test, and run tests side by side without collisions.
- Make it the content
- Turn the winning version into the page's permanent content in one step.
When to use it
- You run Payload on Postgres and want testing without a third-party script or vendor.
- Your editors should run tests themselves after a one-time setup.
- You want results you can audit: the data and the statistics are in your own stack.
Compatibility
| Payload / Payload Postgres adapter / Next adapter / admin UI | >=3.90.0 <4 |
|---|---|
| Next.js | >=16.4.1 <17; App Router |
| React / React DOM | >=19.2.1 <20 |
| Node.js | >=22.23.4 |
| Database | Postgres only (@payloadcms/db-postgres) |
| Postgres requirements | Transactions enabled; pg >=8.13.0 <9 |
| Release | Release candidate on the npm rc tag. Minor releases may change the API. |
| Puck (optional) | @puckeditor/core >=0.24.0; payload-puck >=0.10.3 for slot experiments |
Questions before you install
Is it ready for a stable production dependency?
This plugin is a prerelease. Review the installation, known limitations and upgrade notes before adopting it; plan for migrations and API changes.
Do I need Puck?
No. Puck is optional. You can test Payload blocks, whole pages or registered flag points without it.
Does installing the plugin complete setup?
No. Your app supplies the runtimes, staff access callback, shared rate limiter, proxy, migrations, exposure recording and an authenticated results cron. Follow the full quick start.
What is experiments-stats?
It is the standalone statistics engine: a TypeScript port of GrowthBook’s MIT statistics and assignment code. Its synchronous functions do not collect events or exposures.