Creating an experiment
Compare different paths through an Experience.
Experiments let you compare changes to an Experience and see which one helps customers reach the outcome you want. You might try a different paywall, shorten onboarding, or compare a survey with skipping it.
Nuxie splits customers between variants and measures the results against a plan you set before the test. When you have an answer, you can choose a winner and use it for new arrivals.
Create an Experiment
Ask Nuxie to prepare a test. Describe what you want to change and what would make the change worthwhile. For example:
Compare the current paywall with a version that explains offline access first. Split traffic equally. Measure subscriptions and keep onboarding completion as a guardrail.
Nuxie prepares the variants and the experiment plan. Review the hypothesis, traffic split, and success definition with Nuxie, and try the variant paths in Preview. You can ask for changes before publishing.
Variants and success definition
An Experiment has two to five variants. The control is your baseline, and a treatment contains the change you’re trying. Each treatment needs a complete path through that part of the Experience.
You can also include a holdout. It skips the treatment and continues through the rest of the Journey, which lets you measure the effect of including that section at all. Learn more about variants and allocation.
The success definition says what the test should improve. It includes the main event metric and whether you want it to increase or decrease, along with guardrails for outcomes you don’t want to harm. It also sets the test duration, minimum sample size per variant, minimum improvement, and chance to win.
This definition belongs to the Experiment. The Experience’s optional conversion goal is configured separately, so changing that goal won’t silently change what your test measures. Goals and results covers the settings in detail.
Start, pause, and resume
Publish the Experience
Publish the Experience, then open its Experiments tab.
Start the test
Review the plan on the Experiment card and click Start test. Nuxie begins assigning customers according to your traffic split. They count toward the test when their Journey reaches the Experiment.
Click Pause to send new arrivals to the control variant. Customers already in a Journey keep the variant they started with. Resume returns new assignments to your configured split.
The Experience continues running while the test is paused. If you want to stop new delivery of the entire Experience, use its publishing controls.
Test on a device
Expand Test on a device, enter a customer or device ID, choose a variant, and click Pin. You can then run that path in your app without relying on a random assignment. Pinned customers are left out of the test results.
Delete the pin to return the identity to normal assignment. A pin applies to a new Journey; it won’t switch the variant in a Journey already underway. See Testing in your app.
Restart or copy a test
Choose Restart test from the Experiment’s menu to stop the current run and start a fresh one with the same variants. This is useful when you want a new comparison with its own results.
Copy test creates a separate test with the same variants. Both actions publish the Experience. Nuxie keeps the previous results and doesn’t combine them with the new test.
When you’re ready to end a test and use its winning variant, select Pick winner.