Duelpilot
Start for €0

Guide · PDF, 3 pages

Read your A/B test results honestly. "Not conclusive" is an answer.

Most mistakes in A/B testing happen when reading the result: calling it early, trusting a lucky week, missing what got worse. This guide explains the rules a careful test follows, the way Duelpilot's statistics apply them, and a checklist before you ship a winner.

Get the guide

This creates your Duelpilot account. No payment needed for the guide. Every plan is €0 until you launch your first test.

We email you a 6-digit code; no password. You get a few short setup emails, and every one has a one-click unsubscribe. Privacy

The first page of How to read A/B test results honestly: the cover and why not conclusive is an answer
Page 1 of 3.

What's inside

Six short sections on what a result can and can't tell you, ending with a checklist before you ship.

  1. 1

    "Not conclusive" is an answer

    What a test needs before it says anything, and what to do when it says nothing.

  2. 2

    Peeking: when looking every day is safe

    Why checking a classic test daily invents winners, and how always-valid statistics fix it.

  3. 3

    Sample size: decide before you start

    The planner's numbers, what a detectable change is, and the traffic traps.

  4. 4

    Guardrails and the automatic pause

    What got worse, a broken split, a broken variant: the checks that pause a test on their own.

  5. 5

    Why many tests don't win

    Small changes, ambitious targets, goals far from the change. And why that is fine.

  6. 6

    Before you ship a winner

    A seven-point checklist, and what happens when you ship.

Why an account

Read it, then try it on your own site.

Duelpilot applies these rules for you: verdicts only at full weeks, always-valid statistics, guardrails that pause a test on their own. With your account you can connect your agent right after reading and see them on a test of your own. Autopilot is on Pro and up.

There is no free plan, but there is nothing to pay for the guide, and every plan is €0 until you launch your first test: setting up is free, and your plan (from €9/month) starts when you launch. No launch within 30 days? We cancel it and you pay nothing.

Get the guide

More guides