---
title: "Design tests"
description: "Show your live offer design to some shoppers and a new design to others, then see which one gets more orders to add a protection plan. Start a test, pause or end it, and know when the result is real."
---

Small changes to how the offer looks can change how many shoppers add a protection plan. A design test shows you which design works better in your store, using real shoppers instead of guesses.

A design test (also called an A/B test) splits your shoppers into two groups. One group sees your live design. The other group sees a new design. SureBright then counts which group added more plans. Tests are in [Studio](/merchant-portal/studio), in the [Merchant Portal](/merchant-portal/overview), under the **A/B Testing** tab.

<Note>Design tests come with Studio and are turned on when you ask for them. They work on Shopify only.</Note>

Good to know before you start:

- Only one test can run at a time.
- You need a live design first. Every test compares something against it.
- On the test screens, your live design is called **control**. The design you compare it with is called the **challenger**.

## What you can do

| You want to | Go to |
| --- | --- |
| Try a new design on some of your shoppers | [Start a test](#start-a-test) |
| Take a break, end a test or pick a design | [Pause or end a test](#pause-or-end-a-test) |
| Know which design is better, and when to trust it | [Know when you have a winner](#know-when-you-have-a-winner) |
| Understand which shoppers see which design | [Which design each shopper sees](#which-design-each-shopper-sees) |
| Look at results or older tests | [See results and past tests](#see-results-and-past-tests) |

## Start a test

<Steps>
<Step title="Pick the design to test">
Click **New test**. Under **Pick what to test against**, choose a design from **Drafts** (designs you saved but never made live) or **Previously live** (designs you used before). You can also click **New version** to copy your live design and change it first.

On the right, **What changes** lists every setting that is different between the two designs, side by side. If a design was live before, you also see how it did in its last 30 days.
</Step>
<Step title="Choose how many shoppers see each design">
Under **Traffic split**, pick 50, 70 or 90 percent for your live design, or type your own number. The other number fills in so they add up to 100 percent. The new design can go to anywhere from 10 to 90 percent of shoppers.
</Step>
<Step title="Start it">
Click **Start test**. Studio asks **Start this A/B test?** and tells you what share of shoppers will see the new design. Confirm to start.
</Step>
</Steps>

<Note>
A test has no end date. It starts straight away and runs until you stop it or pick a design.
</Note>

## Pause or end a test

The test card shows **Running** or **Paused**. Your two designs are labelled **Live · control** and **Challenger**. For each one you see:

- **Shoppers**: how many shoppers saw it.
- **Sold**: how many plans it sold.
- **Revenue**: how much money those plans brought in.

Bars underneath show each day. Use the buttons above the figures to switch between **Attach rate** (the share of orders that added a plan), **Policies sold** (plans sold) and **Revenue**, or to look at one kind of plan only.

| You want to | Click | What happens |
| --- | --- | --- |
| Take a break and keep your results | **Pause**, then later **Resume** | While the test is paused, every shopper sees your live design. When you resume, the split starts again. Results so far are kept. |
| End the test and keep your live design | **Stop** | Studio asks **Stop this test?** Every shopper sees your live design. You cannot restart a stopped test, but you can still make the new design live later. |
| End the test and use one design for everyone | **Make `design` live** | That design goes live for every shopper. Studio first asks **End the test and publish `design`?**, or **End the test and keep `design` live?** if your live design is the one ahead. |

After you stop a test, a shopper who already had a page open may still see their design for up to two hours.

While a design is in a running test, you cannot edit it in Studio. Edit a copy instead, or pause and end the test first.

## Know when you have a winner

The better design is the one where a bigger share of orders added a plan. Only orders with a product that a plan can cover are counted. These numbers are measured on Shopify only.

The results page tells you where the test stands in one line:

| You see | What it means | What to do |
| --- | --- | --- |
| **No winner yet** | Neither design is far enough ahead. | Keep the test running. |
| **Early signal** | One design is ahead, but it could still be luck. | Keep the test running. Do not decide on this alone. |
| **Clear winner** | One design is ahead and it is not luck. The lead is shown in points. | Make it live. If your live design won, keep it. |

Only trust a result that says **Clear winner**. It shows up when both designs have had enough orders, and the gap between them is too big to be luck. Early in a test, one design can look far ahead just by chance. That is why it is shown as an **Early signal** and not as a result.

Your live design can win too. If the new design does clearly worse, the result is that your current design stays the better one. In the **Designs** tab, each design shows the result of its latest test, for example **Won 4.6% vs 4.2%**, **Lost …** or **Tie …**. While a test runs, it shows **Testing · … so far**.

You also see the plan money brought in per order that could have had a plan. Two designs can get the same share of orders to add a plan, while one sells more expensive plans. See [Design performance](/merchant-portal/design-performance) for that view and for what each result message means.

## Which design each shopper sees

Each shopper keeps seeing the same design, on every page and every visit, for as long as the test runs. Each order counts for the design that shopper saw, so the two sets of results never mix.

## See results and past tests

Click **View results** to open a test's own page. It shows:

- the one-line result
- a box for each design with its share of orders that added a plan, plans sold and revenue
- a chart over time, which you can switch between those three figures
- the **Pause**, **Stop** and **Promote winner** buttons

When a test has ended and the new design won, the card shows **Publish `design`** so you can make it live straight away.

Older tests are listed under the current one. Each design's card keeps the result of its latest test. If a card says **A test was set up but never started**, nothing is running and every shopper sees your live design. Starting a new test clears it.

<Accordion title="For developers">
On Shopify, the protection plan line in the cart carries a hidden value that links the order to the design the shopper saw. A custom cart or checkout must leave it in place. See [Cart line-item properties](/frontend-widget-integration/frontend-widget-kit/javascript-injection#cart-line-item-properties).
</Accordion>

## Related

<CardGroup cols={2}>
  <Card title="Studio" href="/merchant-portal/studio">
    Make and change the designs you test.
  </Card>
  <Card title="Design performance" href="/merchant-portal/design-performance">
    See how each design is doing and what the result messages mean.
  </Card>
</CardGroup>
