A/B Test a Paywall
Author a controlled paywall experiment and measure its real Experience goal
An Experiment belongs inside an Experience. Users are assigned by the server and count as exposed only when their Journey reaches the Experiment action.
1. State the decision
Start a Task with one uncertainty and the decision the result should inform:
Test whether a 7-day trial increases completed annual purchases compared with direct annual purchase. Keep layout and benefit copy the same so the offer mechanic is the main difference.
Avoid changing several unrelated things in one test unless the combined package is the decision you intend to make.
2. Create the variant paths
In Flow, add an Experiment action and connect each variant to its authored path. Keep a recognizable control. Use a holdout only when “show nothing” is a meaningful baseline.
Check that traffic percentages sum to 100 and that every non-holdout path can reach a valid terminal state.
3. Use the Experience Goal
Put the named Goal after confirmed purchase success. Variant conversion is attributed when an exposed Journey reaches that Goal.
Assignment alone is not an impression. This prevents users who never reached the test from diluting results.
4. Preview each variant
Use Preview and Experiment overrides to exercise every path. Verify Product bindings, cancellation, pending purchase, errors, restore, dismissal, and Goal behavior.
5. Publish with guardrails
Choose Publish and review the Experiment change summary. Adding, removing, renaming, or materially changing variants may require explicit confirmation because it can invalidate accumulated evidence.
Wait for the Version to be Published before starting the test.
6. Start and monitor
Open the Experience’s Experiments tab and start the draft Experiment. Monitor:
- Exposures by variant.
- Goal conversions.
- Conversion rate.
- Sample balance and other result diagnostics shown by Nuxie.
Do not conclude a test only because one variant is temporarily ahead. Wait for the predeclared evidence threshold and check guardrails or contamination.
7. Conclude or continue
When the evidence supports a decision, conclude the Experiment with the selected winner. If the result is inconclusive, keep collecting evidence, stop the test, or create a fresh Experiment with a clearer question.
Restart or clone when a breaking change means new evidence must not be merged with the old Experiment.